I’m teaching a class that uses code to discover unintuitive things about social systems (UC Davis’ CMN 151). One great one shows how hard it is to think about social networks, and it’s easy to state: “On average, your friends are more popular than you” (Feld 1991).
It’s one thing to explain, but something more to show it. I had a demo coded up on Facebook, but it was super fragile, and more of my students use Instagram anyway, so I coded it up again.
To run the demo you
- Consider not to participating (because, for a student, the demo involves logging into your Instagram account on a public computer and running code written by someone with power over you).
- Log in to your Instagram account
- Click to show your Followers, and scroll down that list all the way until they are all showing. This could take a while for people with many followers.
- Open up View -> Developer -> JavaScript Console (in Chrome. “Web Console” in Firefox. Slightly different for other browsers. In Safari you need to find developer mode first and turn it on)
- Ask them to paste the code below, which will be accessible to them via Canvas, into their browser’s JavaScript Console. If Followers aren’t showing, it won’t work. This could also take a while if you have many followers. Keep pasting the last part until the numbers are stable. You computer is working in the background growing the list of your followers’ numbers of followers.
- Open this Google Sheet.
- Paste your values into the sheet.
- Calculate the average number of followers, and the average number of followers of followers. Compare them. With enough participants, the second will be bigger, even if you exclude giant robot accounts.
This post isn’t an explainer, so I won’t get into how and why it’s true. But the way you set it up beforehand in class is by reasoning that there shouldn’t be a systematic difference between your and your friends’ popularities. The numbers should be the same. You wrap the lesson up after the data is in by hopping onto the spreadsheet live and coding up the averages of their followers, and of their friends followers, to show that their friends’ average is higher on averages. After explaining about fat tails, you drive it home on the board by drawing a star-shaped network and showing that the central node is the only one that is more popular than her friends, and all others are less popular.
The code
Open your Instagram Followers (so that the URL in the location bar reads https://www.instagram.com/yourusername/followers/) and paste this into your JavaScript console.
// from https://stackoverflow.com/questions/951021/what-is-the-javascript-version-of-sleep
function sleep(ms) {
return new Promise(resolve => setTimeout(resolve, ms));
}
function instaFollowerCount(page) {
return parseInt(page.querySelector("a[href$='/followers/']").firstElementChild.textContent.replace(/,/g, ""))
}
function instaFollowerCount2(page) {
return parseInt(page.querySelector("head meta[name='description']").attributes['content'].value.match(/([\d,]+)\sFollowers/)[1].replace(/,/g, "") )
}
function instaFollowerList(page) {
return Array.prototype.slice.call(page.querySelector("div[role='presentation'] div[role='dialog']").querySelector("ul").querySelectorAll("a[title]")).map(x => x.href)
}
// https://stackoverflow.com/questions/247483/http-get-request-in-javascript#4033310
function httpGet(theUrl)
{
var xmlHttp = new XMLHttpRequest();
xmlHttp.responseType = 'document';
xmlHttp.open( "GET", theUrl, false ); // false for synchronous request
xmlHttp.send( null );
return xmlHttp.response;
}
function httpGetAsync(theUrl, callback)
{
var xmlHttp = new XMLHttpRequest();
xmlHttp.responseType = 'document';
xmlHttp.onreadystatechange = function() {
if (xmlHttp.readyState == 4 && xmlHttp.status == 200)
callback(xmlHttp.response);
}
xmlHttp.open("GET", theUrl, true); // true for asynchronous
xmlHttp.send(null);
}
var iFollowers = instaFollowerCount(document);
var aFollowers = instaFollowerList(document);
var docs = [];
for (f in aFollowers) {
httpGetAsync(aFollowers[f] + "followers/", function(response) {
docs.push(instaFollowerCount2(response));
});
if(f % 100 == 0 & f > 0) {
await sleep( 1000 * 60 * 30 + 10000); // in ms, so 1000 = 1 second.
// instagram limits you to 200 queries per hour, so this institutes a 30 minute (plus wiggle) wait every 100 queries
// If you're fine running the demo with just a sample of 200 of your followers, that should be fine, and it's also way faster: this demo can run in seconds instead of taking all night. To have it that way, delete the above 'await sleep' line.
}
}
And then, after waiting until docs.length is close enough to iFollowers, run
console.log(`You have ${iFollowers} followers`);
console.log(`(You've heard from ${docs.length} of them)`);
console.log("");
console.log(`On average, they have ${docs.reduce((total, val, i, arr) => total + val) / docs.length} followers`);
console.log(`Your most popular follower has ${docs.reduce((incumbent, challenger, i, arr) => incumbent > challenger ? incumbent : challenger)} followers`);
console.log(`Your least popular follower has ${docs.reduce((incumbent, challenger, i, arr) => incumbent < challenger ? incumbent : challenger)} followers`);
The result isn't meaningful for just one person, but with enough people, it's a strong lively demo. See how things are coming along for others on this Sheet.
Technical details
Instagram crippled their API, so it isn't possible to run this demo above board, not even with the /self functionality, which should be enough since all participants are logged in to their own accounts. This code works by getting the list of usernames of all followers and posting a GET request for that users page. But Instagram can tell you are scraping so it cripples the response. That's why instaFollowerCount differs from instaFollowerCount2. In the main user's page, the followers are prominent and relatively easy to scrape, but the requested page of the friend can't be reached through a console request. Fortunately, Instagram's "meta" summary description of a user's page in the lists their number of followers, so a simple regex yields it. Of course, even scraping the follower count and IDs from the main page is tricky because Instagram has some scheme to scramble all class names for every page load or account or something. Fortunately it's still a semantic layout, so selector queries for semantic attributes like "content", "description", and "presentation" work just fine to dig up the right elements. Of course, this could all change tomorrow: I have no idea how robust this code is, but it works on Oct 24, 2018. Let me know if your mileage varies.
