Instagram Demo: Your friends are more popular than you


I’m teaching a class that uses code to discover unintuitive things about social systems (UC Davis’ CMN 151). One great one shows how hard it is to think about social networks, and it’s easy to state: “On average, your friends are more popular than you” (Feld 1991).

It’s one thing to explain, but something more to show it. I had a demo coded up on Facebook, but it was super fragile, and more of my students use Instagram anyway, so I coded it up again.

To run the demo you

  1. Consider not to participating (because, for a student, the demo involves logging into your Instagram account on a public computer and running code written by someone with power over you).
  2. Log in to your Instagram account
  3. Click to show your Followers, and scroll down that list all the way until they are all showing. This could take a while for people with many followers.
  4. Open up View -> Developer -> JavaScript Console (in Chrome. “Web Console” in Firefox. Slightly different for other browsers. In Safari you need to find developer mode first and turn it on)
  5. Ask them to paste the code below, which will be accessible to them via Canvas, into their browser’s JavaScript Console. If Followers aren’t showing, it won’t work. This could also take a while if you have many followers. Keep pasting the last part until the numbers are stable. You computer is working in the background growing the list of your followers’ numbers of followers.
  6. Open this Google Sheet.
  7. Paste your values into the sheet.
  8. Calculate the average number of followers, and the average number of followers of followers. Compare them. With enough participants, the second will be bigger, even if you exclude giant robot accounts.

This post isn’t an explainer, so I won’t get into how and why it’s true. But the way you set it up beforehand in class is by reasoning that there shouldn’t be a systematic difference between your and your friends’ popularities. The numbers should be the same. You wrap the lesson up after the data is in by hopping onto the spreadsheet live and coding up the averages of their followers, and of their friends followers, to show that their friends’ average is higher on averages. After explaining about fat tails, you drive it home on the board by drawing a star-shaped network and showing that the central node is the only one that is more popular than her friends, and all others are less popular.

The code

Open your Instagram Followers (so that the URL in the location bar reads https://www.instagram.com/yourusername/followers/) and paste this into your JavaScript console.



// from https://stackoverflow.com/questions/951021/what-is-the-javascript-version-of-sleep
function sleep(ms) {
return new Promise(resolve => setTimeout(resolve, ms));
}
function instaFollowerCount(page) {
return parseInt(page.querySelector("a[href$='/followers/']").firstElementChild.textContent.replace(/,/g, ""))
}
function instaFollowerCount2(page) {
return parseInt(page.querySelector("head meta[name='description']").attributes['content'].value.match(/([\d,]+)\sFollowers/)[1].replace(/,/g, "") )
}
function instaFollowerList(page) {
return Array.prototype.slice.call(page.querySelector("div[role='presentation'] div[role='dialog']").querySelector("ul").querySelectorAll("a[title]")).map(x => x.href)
}
// https://stackoverflow.com/questions/247483/http-get-request-in-javascript#4033310
function httpGet(theUrl)
{
var xmlHttp = new XMLHttpRequest();
xmlHttp.responseType = 'document';
xmlHttp.open( "GET", theUrl, false ); // false for synchronous request
xmlHttp.send( null );
return xmlHttp.response;
}
function httpGetAsync(theUrl, callback)
{
var xmlHttp = new XMLHttpRequest();
xmlHttp.responseType = 'document';
xmlHttp.onreadystatechange = function() {
if (xmlHttp.readyState == 4 && xmlHttp.status == 200)
callback(xmlHttp.response);
}
xmlHttp.open("GET", theUrl, true); // true for asynchronous
xmlHttp.send(null);
}
var iFollowers = instaFollowerCount(document);
var aFollowers = instaFollowerList(document);
var docs = [];
for (f in aFollowers) {
httpGetAsync(aFollowers[f] + "followers/", function(response) {
docs.push(instaFollowerCount2(response));
});
if(f % 100 == 0 & f > 0) {
await sleep( 1000 * 60 * 30 + 10000); // in ms, so 1000 = 1 second.
// instagram limits you to 200 queries per hour, so this institutes a 30 minute (plus wiggle) wait every 100 queries
// If you're fine running the demo with just a sample of 200 of your followers, that should be fine, and it's also way faster: this demo can run in seconds instead of taking all night. To have it that way, delete the above 'await sleep' line.

}
}


And then, after waiting until docs.length is close enough to iFollowers, run



console.log(`You have ${iFollowers} followers`);
console.log(`(You've heard from ${docs.length} of them)`);
console.log("");
console.log(`On average, they have ${docs.reduce((total, val, i, arr) => total + val) / docs.length} followers`);
console.log(`Your most popular follower has ${docs.reduce((incumbent, challenger, i, arr) => incumbent > challenger ? incumbent : challenger)} followers`);
console.log(`Your least popular follower has ${docs.reduce((incumbent, challenger, i, arr) => incumbent < challenger ? incumbent : challenger)} followers`);


The result isn't meaningful for just one person, but with enough people, it's a strong lively demo. See how things are coming along for others on this Sheet.

Technical details

Instagram crippled their API, so it isn't possible to run this demo above board, not even with the /self functionality, which should be enough since all participants are logged in to their own accounts. This code works by getting the list of usernames of all followers and posting a GET request for that users page. But Instagram can tell you are scraping so it cripples the response. That's why instaFollowerCount differs from instaFollowerCount2. In the main user's page, the followers are prominent and relatively easy to scrape, but the requested page of the friend can't be reached through a console request. Fortunately, Instagram's "meta" summary description of a user's page in the lists their number of followers, so a simple regex yields it. Of course, even scraping the follower count and IDs from the main page is tricky because Instagram has some scheme to scramble all class names for every page load or account or something. Fortunately it's still a semantic layout, so selector queries for semantic attributes like "content", "description", and "presentation" work just fine to dig up the right elements. Of course, this could all change tomorrow: I have no idea how robust this code is, but it works on Oct 24, 2018. Let me know if your mileage varies.


Change your baby’s astrological sign with physics!

My summer project this year was a little non-academic web app project.

http://whatsyoursign.baby/

The premise of the site is that the mechanism of astrology is gravitational influence, and that since small nearby things have influence comparable to large things far away, it should be possible to tune your child’s astrological sign by giving birth around specifically arranged person-made objects. As a pop science site, you’ll see that it is a pretty soft sell: not telling anyone that astrology is wrong, instead trying to channel the interest in astrology into relevant subjects of physics.

I haven’t even released the site yet, but as a summer project it’s already a big success. I developed my frontend skills a bunch, and learned how to use astrological ephemeris databases. I also learned that astrology has a big open source community. I learned that there is a .baby and .amazon top-level domain for web addresses. I also learned a bit more about how to teach web programming students, hopefully showing the bones of the Internet a bit and making code a bit less intimidating.


Appearance on two podcasts with Steaming Piles of Science

Steaming piles of science is a very fun science podcast based out in New Hampshire. They recorded a “Science pub” I did with colleagues on the science and practice of community building, and we followed that up with a wider ranging sit-down.

Here they are:
https://steamingpilesofscience.com/upcoming-episodes/page/2/

About

This entry was posted on Monday, September 17th, 2018 and is filed under audio/visual.


Tom Lehrer song ripping on quantitative social science

Tom Lehrer was is a cold war era lefty musical satirist, best known for Poisoning Pigeons in the Park, and his jingles about math, science, and nuclear holocaust. In addition to being a musician, he also taught math and stats at MIT and Santa Cruz. His courseload at MIT through the 1960’s included the Political Science department’s quantitative modeling course, an experience that seems to have made him very mocking about the sciences of society. The song below is addressed to sociology but, as he admits, it’s really about all quantitative approaches to social science.

Some choice bits:

They can take one small matrix,
and really do great tricks,
all in the name of socioloigy.

They can snow all their clients,
by calling it a science,
although it’s only sociology.

Elsewhere in the same clip are very nerdy mathematical songs, and a good satire about professors thinking we’re brilliant, and a School House Rock type kids song. Before stumbling on this, I discovered and rediscovered a bunch of other wonderful songs, such as the Vatican Rag, “I got it from Agnes”, and Oedipus Rex. I was especially into Selling Out.


1920’s infoviz, when “Flapperism” was the culmination of Western civilization

HistoryInfoviz_Dahlberg

This image offers a schematic of Western history with a two-axis timeline that brings attention more effectively to long periods. It was published in the journal Social Forces in 1927.

Its author Arthur Dahlberg was a science popularizer and Technocrat active through the 20’s and 30’s. His books, which presented economic systems as closed plumbing systems and other visual metaphors, brought technocratic ideas to many important thinkers in the first half of the 20th century, making him the route by which Technocratic ideas influenced the science of complex systems. Technocracy was a social movement and economic theory that can best be glossed as capitalism under a planned economy. It was popular among farmers and other rural Americans, but was ridiculed otherwise. Nevertheless, its popularity brought it to the attention of people like Herbert Simon, who made fundamental contributions to organization theory, cognitive science, and economics, and Donella Meadows, whose own stocks-and-flows theories of economic system successfully forecasted today’s population growth and global climate change in the 1970s. His influence on original thinkers in the second half of the last century is what piqued my interest in him, and led me to this fun illustration of the state of the art of information visualization in the 1920’s. I love how it all leads to “Flapperism”, which we’ll guess he takes to mean some kind of societal fizzling over.


Two GIFs about peer review, and an embarrassing story …

1)

unnamed

2)

It is common to have your papers rejected from journals. I forwarded a recent rejection to my advisor along with the first GIF. Shortly after, I got the second GIF from the journal editor, with a smiley. It turns out that I’d hit Reply instead of Forward.

At least he had a sense of humor.

About

This entry was posted on Saturday, December 17th, 2016 and is filed under audio/visual, science.


Extra info about my appearance on BBC Radio 4

I was on a BBC radio documentary by Jolyon Jenkins, “Rock Paper Scissors.” The goal of the documentary was to show that this seemingly trivial game is secretly fascinating, because of what we humans make of it. My own academic contribution to that fun claim has been published here and in much more detail here.

Jolyon was a gracious host, but the documentary was released without any word or warning to me, and with rough spots. I’ve got to clarify a few things.

The most important is an error. The show ended with my describing a game in which people “irrationally” herd together and make lots of money. The results of this game were reported faithfully in the show, but the game itself got defined wrong, and in a way that makes the results impossible. Here’s the full game: All of you pick an integer 1 through X. Each person gets a buck for picking a number exactly one more than what someone else picked. ADDITIONALLY, the number 1 is defined to be exactly one more than number X, making the choices into a big circle of numbers. The documentary left that last bit out, and it’s really important. Without anything to beat X, I’m guessing that everyone will converge pretty quickly on X without much of this flocking behavior. It’s only when the game is like Rock Paper Scissors, with no single choice that can’t be beat by another, that you start to see the strange behavior I describe in the show.

Three more things. All of the work was done with my coauthor and advisor Rob Goldstone at IU, who wasn’t mentioned. Second, it’s not accurate, and pretty important, the way that Jolyon implicitly linked my past work to my current employer. The work presented on the show was performed before I started with Disney Research, and has nothing to do with my work for Disney Research. Last, a lot of what I said on the show was informed by the work of Colin Camerer, specifically things like this.

About

This entry was posted on Wednesday, August 5th, 2015 and is filed under audio/visual, updates.


Enfascination 2013

29742_396066756605_704462_n“Nothing in the world is more dangerous than sincere ignorance and conscientious stupidity.” Thus spoke Martin Luther King Jr. in a great endorsement for humility, curiosity, and discovery.

On Thinko de Mayo, from 1PM, you will have five minutes to help us see how dangerous we are. You may share anything at all during your five minutes, as long as you personally think it’s fascinating. Your goal is to transmit your sense of fascination to others. FB page: https://www.facebook.com/events/498466006869981/

If the constraints of themes help you brainstorm, try “Science towards nescience.” But generally, you should trust yourself. If you manage nothing more than five minutes of wobbling, inarticulate, ecstatic blubbering then Well Done: You have successfully expressed the unfathomable depth of your subject.

This is the ten-year anniversary of these lectures –– ten years since I attempted the world’s nerdiest 21st birthday kegger. This will be the fifth and probably last in Bloomington. Ask me for help if you’ll have slides or a demo.

Past topics have included:
Slide Rules, Counting the Permutations of Digit Strings, Conceptions of Time in History, Chili Peppers, How to cross a glacier, The Singularity, Indiana Jones, Rural desert water distribution systems, Hexaflexagons, Small precious things, Wilderness Camps as Commodity, DIY Cooking, Roman Emperor Deaths , Joy of Science, Salt , Three Great Banquets in Italian History, How to Sharpen a Chisel, Some Properties of Numbers in Base Ten, The Physiological Limits to Human Perception of Time, Geophagy, Pond Ecology, Superstition: For Fun and Profit, Counterintuitive Results in Hydrodynamics, The Wolof Conception of Time, Arctic String Figures, The Seven Axioms of Mathematics, Dr Seuss and his Impact on Contemporary Children’s Literature, Twee, Motorcycle Life and Culture, Cultural Differences Between Japan and the US, Brief history of the Jim Henson Company, Female Orgasm, Insider Trading: For Fun and Profit, Film of Peter Greenaway, A Typographical Incident with Implications for the Structure of Thought, Cooperative Birth Control, Tones in Mandarin, Unschooling and Deschooling, Q&A: Fine Beer, DIY Backpacking, Chinese Nationalism in Tibet, Biofuels, The Yeti, The Health Benefits of Squatting, The Big Bang, How to Pick Stocks Like a Pro, Food Preservation Technique, or Managing Rot, Infant Visual Perception, Demonstrations in Number Theory, Rangolis, Kolum, The Hollow Earth, Edible Mushrooms: For Fun and Profit, Human Asexuality, A History of the California Central Valley Watershed, An Account of the Maidu Creation, The Paleoclimatology of the Levant, Rural India, German Compound Words, Manipulating Children, Physics of Time, Animal Training on Humans, Constructed Languages, This Week’s Weather, The XYZs of Body Language, Light Filtration Through Orchards, Our Limits in Visualizing High Dimensional Spaces,Twin Studies.

Last year’s audio:
http://enfascination.com/weblog/archives/301
And video/notes from before that:
http://enfascination.com/wiki/index.php?title=Enfascination_2011#Enfascinations_Past

pow!
seth.

UPDATE post-party

Here is what happened:

  1. The Tiger Café by Ronak
  2. Jr. High School Poetry Slam by Lauren
  3. The “Border” language by Destin
  4. Perception/Objectivity by Paul Patton
  5. Readings from James Agee by Jillian
  6. “A signal detection theory of morality” or “The morality manatee” by Seth
  7. Dreams and the four candies by Danny
  8. Pick Two by Adam
  9. Trust and Trust Experiments by Jonathan

Rock-Paper-Scissors ms. on Smithsonian, NBC, and Science Daily blogs.

This is my first (inter)national press, I’m a little embarrassed to feel excited about it. Its also a pleasant surprise, I wouldn’t have expected it to have general appeal.

Links:


Enfascination 2012 audio

moby1

Some things take time, but it only takes an instant to realize that you have no idea what’s going on. This epiphany-every time it happens-is punctuated by the sound of 500 stars around the universe literally exploding, dissolving their planets and neighbors in flaming atoms, in silence. It happens every instant, forever. As right as you were, its impossible for you to know how right.

The 2012 program from May 5, 2012, featuring:

  • “Hoosier Talkin’,” Sarah on the southern Indiana dialect
  • “A brief history of Western art” by Eran
  • “Introduction to conducting” by Greg
  • “Infant perception” by Lisa
  • Poems read by Jillian
  • “The paleoclimatology of the Levant” by Seth
  • “Tweepop” by Robert
  • “Direct perception” by Paul Patton
  • “Slide rules” by Ben

“Seth Frey the Sandwich Guy”

In high school I would bring very large sandwiches constructed with many pounds of meat and bread. They were famous enough that I would sell them. I bought some in exchange for people’s souls. Jonathan Lazarus wouldn’t sell his, but he offered to make me a song instead, and I couldn’t have hoped for more.

link
This is 1998 or 1999. He died two weeks ago, hit by a train. His brother Ben made this video to remember Jon. RIP Jonathan Lazarus. RIP also Sean Emdy who was in the same high school class, and was killed the same way in 2003 or 2004. Thanks also to Ben.


My Awe Talk: Inventors who were killed by their own inventions

Awe Talks are a 5-minute fun lecture series started by my pal Kyle. He asked me to record one, here: http://vimeo.com/59541529


Enfascination 2012

Some things take time, but it only takes an instant to realize that you have no idea what’s going on. This epiphany—every time it happens—is punctuated by the sound of 500 stars around the universe literally exploding, dissolving their planets and neighbors in flaming atoms, in silence. It happens every instant, forever. As right as you were, it’s impossible for you to know how right.

Enfascination is a very tiny event that celebrates the act of being
caught. You have five minutes to share something that you think is
fascinating—that’s the only rule. You will find that the people you
are sharing with are fascinated too, and you will be caught by things
you’ve never thought to catch.

The 2012 Enfascination Lectures
Why: I would love for you to share.
When: Saturday, May 5th, or “Thinko de Mayo,” starting at, say, 5PM.
Where: Probably in the basement of Woodburn Hall, on the IU campus
Really?: Probably, maybe not. I just made this all up now so times and places can change. Check this webpage for updates.

This year’s occasion is my 30th birthday, but this is the ninth year that I’ve been hosting this birthday lecture series. Past topics have included Counting the Permutations of Digit Strings, Conceptions of Time in History, Chili Peppers, How to cross a glacier, The Singularity, Indiana Jones, Rural desert water distribution systems, Hexaflexagons, Small precious things, Wilderness Camps as Commodity, DIY Cooking, Roman Emperor Deaths , Joy of Science, Salt , Three Great Banquets in Italian History, How to Sharpen a Chisel, Some Properties of Numbers in Base Ten, The Physiological Limits to Human Perception of Time, Geophagy, Pond Ecology, Superstition: For Fun and Profit, Counterintuitive Results in Hydrodynamics, The Wolof Conception of Time, Arctic String Figures, The Seven Axioms of Mathematics, Dr Seuss and his Impact on Contemporary Children’s Literature, Motorcycle Life and Culture, Cultural Differences Between Japan and the US, Brief history of the Jim Henson Company, Female Orgasm, Insider Trading: For Fun and Profit, Film of Peter Greenaway, A Typographical Incident with Implications for the Structure of Thought, Cooperative Birth Control, Tones in Mandarin, Unschooling and Deschooling, Q&A: Fine Beer, DIY Backpacking, Chinese Nationalism in Tibet, Biofuels, The Yeti, The Health Benefits of Squatting, The Big Bang, How to Pick Stocks Like a Pro, Food Preservation Technique, or Managing Rot, Demonstrations in Number Theory, Rangolis, Kolum, The Hollow Earth, Edible Mushrooms: For Fun and Profit, Human Asexuality, A History of the California Central Valley Watershed, An Account of the Maidu Creation, Rural India, German Compound Words, Manipulating Children, Physics of Time, Animal Training on Humans, Constructed Languages, This Week’s Weather, The XYZs of Body Language, Light Filtration Through Orchards, Our Limits in Visualizing High Dimensional Spaces,Twin Studies. There is video for some of it, notes for others, collected here.

see you there,
seth.