Stop measuring community engagement

The profit motive is being blamed here for optimizing for engagement.<p>But how do you assess progress when you remove all desire to monetize?<p>I run r&#x2F;WallStreetBets and &quot;quality&quot; is an extremely nebulous term.<p>I look for things like &quot;novelty&quot;, &quot;thought-provoking &#x2F; well reasoned commentary&quot;, &quot;original content&quot;, &quot;authenticity&quot;, &quot;self-awareness&quot; or &quot;primary research&quot;. But these are human assessed metrics.<p>Some more easily measurable metrics might include, &quot;length of submissions&quot;, or &quot;number of outbound links excluding blacklisted domains&quot;. Or even &quot;number of tickers or quality-correlated keywords mentioned&quot;.<p>All these metrics have very clear downsides, and if generally well-known, become useless. Interestingly, a score too high can also result in something being unlikely to be authentic.<p>Another challenge is your relationship with users. Surprisingly, moderators are not innately adversarial to users, they can also promote content through other channels (discord, twitter) or sticky threads for a viewership boost.<p>So, even without a profit motive, what do you do?

Good article, though I think it missed explicitly making the point that the <i>reason</i> &quot;engagement&quot; is the metric is because that&#x27;s what platforms <i>monetize.</i> Engagement is a euphemism for attention, and social platforms exist to sell your to advertisers.<p>Since the article is written for &quot;community builders,&quot; understanding that engagement is a metric to quantify the advertising potential of a platform should make it obvious that people who <i>aren&#x27;t</i> in the business of selling attention to advertisers don&#x27;t <i>need</i> to copy social media and optimize for engagement; instead they should target better measurements of value creation for their community. Of course Goodhart&#x27;s law[1] still applies.<p>1: <a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Goodhart%27s_law" rel="nofollow">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Goodhart%27s_law</a><p>(edited for readability)

I enjoyed the Monsters Inc reference and the turning of it on its head central to this piece. But we&#x27;ve said&#x2F;argued this for years and at the end of the day engagement = ad reach = profits and until the economics there change any for-profit social media will gravitate towards engagement. Hell ANY media gravitates toward engagement and outrage bait. An actual community built around Mastodon seems possible, and there&#x27;s community discussions around private group chats but they will always remain niche and not profitable.<p>I don&#x27;t get what <a href="https:&#x2F;&#x2F;savannahhq.com" rel="nofollow">https:&#x2F;&#x2F;savannahhq.com</a> is and what the value add is there but that seems to be what he&#x27;s interested in, niche + small communities.

The article dances around a topic that is very relevant across technology companies- toxic metrics.<p>I worked for many years at a company that prides itself on being metrics driven. To the point that they celebrated having many significant figures after the decimal point when measuring length of negative events in seconds.<p>The good side of metrics driven culture is that having a metric gives you a concrete goal.<p>The downside is that really good metrics are often very hard. By really good, I mean metrics that measure the desired outcome.<p>It’s much easier to measure, for example, how many operation X your service performed, rather than the value the service delivered to your business or your customers. So we assume that each operation X contributes a uniform, positive quantum to those outcomes and we count X’s.

Nicely written post but can&#x27;t help noticing the elephant in the room being tiptoed around, which really makes me question the sincerity of the author.<p>The monsters need energy to power their civilisation. Why do social media companies measure engagement? What are they extracting? There is not one single mention of the &quot;p word&quot; in the article.<p>The last two sections are delivered with such heavy doses of naivety that this ends up just coming across as Orwellian in tone.<p>&gt; <i>“We should measure what we value, not value what we measure”</i><p>Who is &quot;we&quot;? Is &quot;we&quot; the shareholders? What do &quot;they&quot; value do you think?

Early on in a community, there often aren&#x27;t enough data points to cover other possible metrics. With that in mind, starting somewhere is better than not starting at all — measuring engagement is a useful way to begin understanding what&#x27;s resonating in your community. As you continue building your community, there are many different angles that you should use to evaluate its health.<p>Measuring engagement then becomes one piece of data that&#x27;s valuable, but shouldn&#x27;t be the only piece. When engagement data is combined with qualitative community surveys, enthusiasm from members to contribute to a community, clear and timely responsiveness to community needs, membership growth over time and geographies, depth of member interactions (whether across community channels or within specific channels), what&#x27;s topically important to members and why, and overall sentiment and change in sentiment over time - that&#x27;s when community builders can begin to better understand the health of their communities and their impact on their business.<p>Engagement is an important piece, but just one of the pieces, that helps paint the full picture.<p>Disclaimer: I’m a co-founder of Common Room [1] and a we’ve invested a lot of energy in solving for this exact problem. You’re welcome to check out the product (it’s free to sign up) and would love to hear of ideas or feedback.<p>1: <a href="https:&#x2F;&#x2F;www.commonroom.io&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.commonroom.io&#x2F;</a>

Hot take: stop building communities around everything.<p>People are highly tribalist and banding together like that produces a lot of ugly outcomes. Communities form and likely won&#x27;t go away, particularly the more people internalize them emotionally and integrate them into their identity.<p>It is exhausting to see a community for literally everything, even the smallest products.<p>Perhaps we should let products simply exist.

This sounds great, but I see a challenge. The author recommends measuring e.g. user happiness instead of engagement. How to measure user happiness? IMO, some type of survey could be used. But the “best” metrics are based on observable behavior. Not what users self-report, because they may not know how they actually feel about the experience. In general it’s better to see what users do rather than what they say.

Unfortunately what is really needed here is a suggestion of something else to measure that is better. I fight this a lot with bad metrics (code test coverage), nobody wants to give them up unless I can suggest a replacement to measure.

Couldn&#x27;t agree more.<p>I believe this falls into the &#x27;Results Oriented&#x27; mindset of delivering value. This means figuring out your desired outcome ahead of time, then measure it over time. In addition, overtime, you may determine that your desired outcome has changed and you need to measure differently. Yes, iterating is not just for software.<p>IMHO pure quantitative metrics are of little value for customer experience. I believe metrics should be a little squishy and subjective. The value in the subjectivity is that it spurns conversation and actual thought instead of simple counting.

That&#x27;s a great post, and I plan to share it with others.<p>Sadly, it&#x27;s pretty difficult to measure for some of the &quot;values&quot; that my communities run on, but &quot;engagement&quot; isn&#x27;t even on the map.<p>I suspect that &quot;engagement&quot; does, indeed, provide real monetary value for advertising-based sites, as clicks == money. Since that isn&#x27;t a factor in my communities, we don&#x27;t worry about that. In fact, we try to minimize &quot;engagement.&quot;

I don&#x27;t get it. What is a &quot;community manager&quot;? Is it someone that runs a company&#x27;s social media page? And do they think that people care about their brand so much that they would join a community of people whose only common trait is liking the same brand? And in the first place, how do you even build a &quot;community&quot; manually and artificially? It&#x27;s almost a bastardization of the word, as if a community is not a set of people with similar goals but instead yet another way to convert people&#x27;s time and desire for social interaction into profit.<p>The article implies that community building should be some kind of altruistic purpose, where your only goal is to maximize the amount of &quot;meaningful relationships&quot; created? But building an artificial community in the first place can never be altruistic because the end goal of it all is to guide people to your product or conference or whatever. If you were an actual altruistic community builder, you would be telling people to get off the Internet and go make &quot;meaningful relationships&quot; with people in real life.

This article completely undermines itself by arguing in the conclusion (&quot;Measure what you value&quot;) that we can and should measure &quot;meaningful relationships&quot; instead.<p>&gt; <i>They’re not as easy to measure as engagement, sure, but they can be measured. </i><p>I&#x27;m afraid that a citation is sorely needed here. There are no real measurements for &quot;meaningful relationships.&quot;<p>In fact, the only proxy metric we have for &quot;meaningful relationships&quot; is engagement!<p>And, yes, engagement is not a very good proxy metric for meaningful relationships, but since there&#x27;s no alternative, it&#x27;s all we&#x27;ve got, so this article is pointless.

Here&#x27;s the little discussion I have in my head when this issue comes up.<p>&quot;Engagement isn&#x27;t a good metric. Optimize for having a good community.&quot;<p>&quot;But engagement is easy to measure AND it means people are using the product. If people are choosing to use the product, doesn&#x27;t that mean the product is giving them value?&quot;<p>&quot;Not necessarily. Drugs like tobacco and heroin have fantastic &#x27;engagement&#x27; but we consider them bad for us.&quot;<p>&quot;But how do you tell the difference between something that engages because it&#x27;s good and something that engages because it&#x27;s some kind of drug?&quot;<p>&quot;I guess one important signal would be when people know something is bad for them but they&#x27;re doing it anyway?&quot;<p>&quot;Is that the case for social media? Do people feel addicted - feel that it&#x27;s bad for them but they can&#x27;t stop?&quot;<p>I think we&#x27;ve heard a lot of anecdata on this last question, but I can&#x27;t recall seeing any big survey on it.<p>EDIT: according to this, people will pay to have their social media restricted: <a href="https:&#x2F;&#x2F;www.washingtonpost.com&#x2F;outlook&#x2F;2021&#x2F;07&#x2F;19&#x2F;social-media-addiction-social-science&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.washingtonpost.com&#x2F;outlook&#x2F;2021&#x2F;07&#x2F;19&#x2F;social-med...</a>

I think people enjoy this argument because it&#x27;s counter intuitive, but I also think it&#x27;s counter intuitive because it&#x27;s wrong. If you build a playground and everyone avoids the teeter-totter, there&#x27;s probably something wrong with the teeter totter. Oddly the monsters inc example seems counter to his point since it seems like they were measuring something to specific and the proper solution would be to something more general (like engagement) instead. It seems to me like trying to start with something other than engagement would be premature optimization.<p>Of course engagement is insufficient and of course if you want to know something else, measure that other thing, and of course you should interrogate your metrics, and maybe it&#x27;s just this thing that bugs me about modern writing where you have to take this ridiculous extreme stance to get eyeballs but yeah.

Maximizing community engagement, trolling and advertising are related. They&#x27;re different heads of the same multiheaded demon.<p>All 3 are concerned with attention. Are hungry for it.<p>If social media could be optimized to deliver engaging conversation to everybody then the demon would be slain. Because the hunger would be satisfied.<p>Does that sound right?

So please tell me, what should I optimize for when moderating a local OpenStreetMap community. I personally feel it&#x27;s crucial people internalize that&#x27;s a project with a bottom-up organization. A &quot;civil society&quot;, if you will. Alas, despite 200-250 daily active mappers in Poland [1] only a fraction ever posts on the forum, even if just to ask something. Meeting IRL is also futile except 2-3 largest cities. We have a forum, Facebook group and Discord.<p>Whatever I&#x27;d come up with, I&#x27;d like it to be a self-healing organism that would continue to thrive even if some important members become inactive. And to feel it&#x27;s not a Sisyphean task.<p>[1] <a href="https:&#x2F;&#x2F;osmstats.neis-one.org&#x2F;?item=countries&amp;country=Poland" rel="nofollow">https:&#x2F;&#x2F;osmstats.neis-one.org&#x2F;?item=countries&amp;country=Poland</a>

&gt;Now imagine if this played out the other way around. Imagine that the monsters originally focused on maximizing energy by making children laugh as much as possible. Then one day, in a horrible twist, someone discovers that a child’s scream was a much easier and more powerful way of meeting their goals. Their company, which built a platform that gave them instant access to children around the world, suddenly realizes that they could reach their goals by spreading fear rather than joy.<p>Then they would instantly start using screams? Nobody making business decisions in social media companies will prioritize &quot;user happiness&quot; over increased production of their main product (your attention) for their customers (advertisers).

I like the premise of the post. I think it paints a rather overly idealistic view of social networks, who exist to make money first, not to create value (if they could make money without creating value, they certainly would do that).

It’s because investors aren’t discerning either<p>They chose to pay higher prices for shares, divorced from actual revenue<p>The companies, in-turn, found non revenue metrics to show quarter over quarter<p>and thousands of startups copied the model<p>Just don’t rely on the A&#x2F;B test, be aware of it, but don’t rely on it. Put the human control back into the process. Users aren&#x27;t staying on your platform 5 seconds longer after you hid the escape hatch because they love it, they are frustrated and lost! The A&#x2F;B test doesn&#x27;t tell you why the engagement is longer, only that it is. It takes an empathic human to say “okay, B got us that result but not for the reason we want”

<i>Even though it’s the energy they’re after, not the screams, it’s the screaming that was their primary metric, and everything about their company was organized around increasing that.</i><p>Missed opportunity to bring it full circle back to this.<p>Otherwise, not bad.

As an end user, fighting back against using anger to drive engagement is easy.<p>Stop engaging with content that angers you.<p>Don&#x27;t follow politicians and news outlets you despise. If someone you follow retweets the latest garbage being spewed by Fox News, don&#x27;t interact with it. No retweets, no replies, no likes, no matter how well they refute the bullshit or how well they &quot;own&quot; the original messenger.<p>The algorithms feed you content they think you&#x27;ll engage with. Train the algorithm to feed you better content.

Lately - it seems we&#x27;re all caught between 2 worlds:<p>1. If you over focus on impact the quality of your product gets torn to shreds. You&#x27;ve built great direction but do people really use&#x2F;like your product?<p>2. You build for engagement, people use the thing, but what impact did it drive?<p>This article is great, but it&#x27;s unbalanced potentially. I think directionally, some average in the middle is likely the way - so why not both? And while we&#x27;re add it, add in other indicators. Success metrics that balance all of that, plus just a vibe metric for good measure.

&#x27;Stop making decisions based on engagment measurement&#x27; might be a better approach, if what you care about is maintaining the quality of your online social group. Some social media sites have tons of high-quality original content submitted by people with expertise in their domains, and some are 90% fluff. If you want to attract people who&#x27;ll make high-quality contributions, getting rid of the fluff is the most important goal.

engagement takes many forms. If you value people connecting with each other, measure how many messages they&#x27;re sending or connections they&#x27;re making.<p>Saying stop measuring community engagement is a good clickbait headline, but it&#x27;s absurd advice for community managers that even your own article doesn&#x27;t agree with.

Is engagement really value neutral?<p>In the extreme, if you spend all your time &quot;engaging&quot;, something toxic is happening

I wonder if measuring engagement and optimising for engagement should be separated. I’m glad that there is lots of engagement on HN but I’m also glad that HN chooses not to implement a number of engagement optimisation features as well as downranking topics that result in high engagement flame wars.

when community engagement is measured badly, its because community is being treated as a product like any app where MAU and time on site is being prioritized, yet companies dont dedicate even 1&#x2F;100 of the resources to community “product” that they do their software product (perhaps because it is not viewed as a real job, as perceived in other comments here)<p>that said, you do need a way to justify to others why you are a better community manager than the next, and whether you are doinng a good job. that part went unanswered here and the article would be better if it had a constructive replacement instead of just arguing to be unaccountable to the rest of the org.

If you can find a way to easily quantify the quality of human social interactions, you&#x27;re probably on the short list for a Nobel prize. It&#x27;d have applications in every field related to humans.

This is an issue with more than just the tech industry. It’s an issue with modern society itself.<p>Take the GDP, the number that when it’s first derivative goes negative, we shout “recession” and use it as an excuse for all manner of harmful (and sometimes beneficial) behaviors. What is GDP? It’s not a measure of wealth; it’s a measure of production.<p>Take the boots paradox, for example. A society making $50 boots that need replacing after a few months, will have a higher GDP than a society making $100 boots that last for years. And the shitty boots society, despite spending more on boots - to paraphrase Terry Pratchett - will still have soggy feat.<p>It’s a hard problem, and I can’t see any obvious way to solve it. Both in social networks, and in economics. For whatever reason, consumption seems to be much easier to measure than wealth.

Twitter isn&#x27;t a community, it&#x27;s a platform. There&#x27;s nothing at all wrong with community engangement. Platform engagement and the advertising model that requires it are the problems.

Advertising is a kind of trolling. I think that&#x27;s an important connection to ponder. It could lead to something.

But... How? It&#x27;s missing how to do it (measure value) in the article.

&gt; Originally they were there to connect people and let them share in each other’s joy.<p>No, they looking for a ways how to easily and faster to fuck.<p>&gt; They couldn’t measure joy, but they could measure engagement, and so they did. &gt; That measurement became their goal, and they focused on maximizing it.<p>Because this is social networks profit. Ads, and more people attached to some topic where can be shown an ads to these people. That&#x27;s why and only that&#x27;s why it is works like that. Nobody cares about principles. Only money. Power in money, a money is a power.<p>&gt; What they didn’t understand was that negative, divisive, hate-inspiring posts get more engagement than positive, supportive, kind posts do.<p>For what do your country has intelligence and different defense structures? Because it is very easy manipulate by people and do wars. Extremely easy to organize that and involve a lot of people to mass destruction events. Extremely easy. And i do not know (the same as you too) who is behind some kind of negative behind.<p>&gt; . The algorithm didn’t care whether the posts brought joy or anger, it only cared about whether they brought engagement.<p>Because this is about ads and money. More people will watch ads more money social network will get. Easy.<p>&gt; So in order to maximize engagement, the algorithm actively encourages and elevates posts that cause unhappiness among the platform’s users.<p>You said that? Is a bots with fake votes, SEO thing is not about manipulating data, opinions? Its easy to make fake accounts and promote hate as a wide interested topic, while it&#x27;s not. Only social network know that.<p>&gt; They join looking for support, or connections, to give back, or find meaningful relationships.<p>No, they looking for themselves reflected in the community.<p>&gt; Measuring engagement doesn’t tell you if your community is happy, or healthy, or even accomplishing any of the things you were hoping it would accomplish.<p>Depends on a case. Some measuring engagement can prevent negative business processes.<p>&gt; Knowing how much engagement you have doesn’t tell you how much value you’re getting<p>For that you have another metrics. Depends on a case. Registration, profit, grow, etc. Final countable results.<p>&gt; Let’s bring this back to community building. We still want our members to be happy, right?<p>Who the fuck am i to let people happy? Happiness is subjective thing. I have no clue, no ability to make someone happy. The same as you, the same as anybody else. This is very agile thing depends on many factors and the most important wish to be happy and objective situation.<p>&gt; So what should we value?<p>Does your community member even read or know about &quot;values&quot; at all? Is it some kind of cult, or what?<p>----<p>I have a well known secret. To get maximum engagement need to say that the white color is black color. Water - is not a liquide, and planet earth is flat.<p>And then your &quot;engagement&quot; will go above the sky. Just always need to split people, and say that one thing is not the thing what you used to think, it&#x27;s another thing. totally vise versa.<p>Oh... Darling, exactly this you have done by saying: &quot;stop measuring engagement&quot;. And by this title invited to engagement a lot of people who forced to read all of your nonsense to one more time confirm that white is white, black is black.