Online actions with offline impact: how online social networks influence online and offline user behavior Althoff et al., WSDM 2017
You can go to a lot of effort to build social networking features or support into your app or website. If the goal is engagement directly within the app then at least you have something you can measure, even if cause and effect can be hard to untangle . But if the goal is to drive user behaviour outside of the app this becomes a much harder problem. An e-commerce example would be the desire to drive foot traffic into retail stores (see e.g., Facebook’s dynamic ads for retail). A more direct example is fitness apps want to drive physical activity outside of the app. In today’s paper, Althoff et al. analyse a dataset uniquely qualified to help us get a handle on the benefits of social networking features as they relate to changing real world (‘offline’) behaviour. The headline results suggest the social features are worth it: creation of new social connections leads to a 30% increase in in-app activity, 17% improvement in user retention, and 7% increase in real-world activity.
Many successful websites and apps use social networking features to appeal to their users, allowing them to interact, form social connections, post updates, spread content, and comment on other’s posts. Social networking features are ubiquitous and are not only used by online social networks, such as Facebook and Twitter. For example, news reading, online education, music listening, book reading, diet and weight loss, physical activity tracking, and many other types of modern computing applications all heavily rely on social networking… However the impact of the online social networks on user behavior remains elusive.
The dataset they study contains data on 791 million online and offline actions of 6 million users in a physical activity tracking app over the course of 5 years. This is four orders of magnitude more users, and six orders of magnitude more activity than datasets used in previous comparable studies on social network effects on physical activity. The following features also make the dataset ideally suited for study:
- There is no need to rely on self-reporting of activity (which is known to be unreliable) as the app tracks actual steps. Thus it becomes possible to reliable compare activity posts as a measure of online engagement, and steps taken as a measure of offline physical activity behaviour.
- The social networking features were first introduced 2 years after measurements began, so before and after social network introduction comparisons are possible
- The network contains both friend and follow actions. Friend requests require approval – the delay between a friend request being made and it being accepted proves especially useful in teasing out social effects. After a friend request is accepted the new friend’s activities appear in the user’s timeline and commenting is allowed.
The key idea of our approach is to identify a natural experiment where we control for the amount of intrinsic motivation and vary the effect of social edge formation. We achieve this by relying on the fact that social network edges only become active (i.e., expose users to notifications and posts), after being accepted by the target person. This means we can compare the population of users who all sent edge requests (i.e., they were all intrinsically motivated) but some of these requests get accepted immediately, while others get accepted with significant delay.
The study compares the difference in activity levels between 7 days before a friend request, and 7 days after. We can see how this varies between users whose friend requests were accepted immediately (within one day), and those whose friend requests were not accepted within seven days (8% of all friend requests, with a median time to acceptance of 25 days).
Untangling intrinsic motivation and social influence
Even when a friend request is not accepted, users sending such requests increase their average daily activity by 148 steps (
When the friend request is immediately accepted though, average daily activity increases by 328 steps (
This means that the difference of 180 average daily steps can be attributed to an influence effect I from the social connection. This influence effect I is statistically significant (95% confidence interval between 74 and 236 steps; p < 10-3; Mann-Whitney U test).
Exposure to the new friend’s activity and status updates leads to higher physical activity.
How does joining a social network impact user behaviour?
What impact does the very first edge of a user have? I.e., what is the effect of joining the network for the first time.
Since the social network was introduced only after two years into our observation period, we can measure the effect its introduction had on user behavior. We show that joining a social network makes users more physically active for a period of several months, increases their number of posts within the app, and makes them more likely to continue to use the app compared to users in a matched control group.
The study compares paired users showing very similar activity levels for 4 weeks before one of them joined the social network, and the other one didn’t…
Joining the network gives an immediate 7% boost in activity, and the effect then gradually diminishes over a 20 week period – after which the two groups are statistically indistinguishable again.
Do social network features increase online engagement and retention?
A similar matching technique based on number of activity posts per day is used to compare users who started using the app around the same time (within 7 days of each other), but where one user started using the social network and the other didn’t.
Users who do join the network are 17% more likely to be retained after one year. Joining the network also gives a short term boost in in-app (as opposed to physical) activity, but this tails off over the year until the effect is indistinguishable.
What impact does adding each new friend have?
Perhaps unsurprisingly, the more friends you already have, the less the marginal benefit of adding an additional friend. The authors studied this effect for the first 5 friends:
It’s all quite predictable really
Finally, the authors show that it is relatively easy to predict behaviour change in the dataset using a combination of features including activity levels, type of social edge created, and user demographics (age, gender, BMI). The model can predict which users will be influenced to increase their activity after each creation (classifier) with high accuracy (78% AUC).
