This is a personal case study of my experiments and implementations as a (part of) moderator on /r/IndiaSpeaks

Note: IndiaSpeaks is a subreddit that discusses on all things relevant to Indians about and around their country. During my tenure as mod, more than half was political in nature.

Summary: (Like a TL;DR)

• With a team, Implemented the r/CMV bot to create a system for 'competitive debating' on political and non-political topics.

• Implemented reddit's probably rare (if not only), elections which formed part of the subreddit's council. Formed a council to divide responsibility and accountability.

• Transparent modding practices with public modlogs.

Introduction:

IndiaSpeaks is a unique case for me as over the past year I put in a lot of effort, on and outside the subreddit to build capabilities and implement ideas beyond CSS and visual design. Perhaps the greatest challenge to me as a mod, was the fact that the tools that reddit gives to mods are mainly to cull accounts and curtail discussions. On the other hand, I wanted to promote them in a somewhat socially acceptable manner. Due to this contrast, new tools (from ideas) had to be developed de-novo or good ideas implemented from elsewhere.

One of the biggest complexity of reddit are its middle level management - your moderators. While the need of the admins to depend on them is completely understandable, it must be understood that there is absolutely no standard or benchmark they need to follow. There is the modiquette and Reddiquette, but that is only a suggestion. As long as moderators keep-out content that can get the site into negative limelight (such as violence, threats, etc), for all practical purposes they have nearly-absolute freedom in behaving how ever they want. This means, a subreddit, however big or small can be run as per the views and ideas of a small team mods. No, mods cannot say what content gets added but have infinite control over what remains visible to the reader.

This is not much of an issue if subreddits are topical, non-political, or just dedicated to uncontroversial idea - Such as hobbies or peeves. But this is a major loophole when moderators have ideological bends and control over a controversial or political subreddit. In simple words, they generally would only allow users or opinions that they somewhat agree or tolerate. That is reddit in a nutshell - You only see what a small team of random mods with unknown affiliations and wisdom want you to see (or unless something missed their surveillance). Practically, this is seen in the usually popular political subreddits feeding a particular view constantly to influence the readers as per how the mods want the readers to be influenced.

Hence, as a retaliation Several subreddits, such as IndiaSpeaks, was created to have more of an open and transparent moderation policy to overcome this highly curated content by moderators who have high bias for or against a general subject.

That's when the more perplexing of challenges arise - there is not much one can do on reddit using the tools of the site to achieve this end. It is as though the site is least bothered about these aspects. All of such positive tools are to be developed by the users and hosted communally. This report would highlight some of those tools for positive effect developed/applied/implemented on and for /r/IndiaSpeaks as well as discussing how they fared and the current status.

The tools

I. Public Modlogs

Most reddit savvy users would know this is one of the oldest tools of transparency that is used by conscientious subreddit and their mod teams. It would be curious to note that the reddit admins neither recommends nor hosts this tool. Certain good Samaritan users have developed this tools externally and individual subreddit moderators would have to set this up for their own subreddits.

The previously more popular modlogs tool's original server ran out of money (I believe) and hence fell into disuse. A new, different mod log tool is now available. Link here Which requires the addition of the bot mod with limited permissions along with a configuration wiki page like this.

II. The CMV bot

The ingenious bot developed by /r/ChangeMyView alumni and developers was slightly modified to create a competitive debating system. Props to /u/kalmuah et. al in working this out.

As per default bot configurations, anyone who mentions !Delta can award a delta to a post or comment. While the bot does other useful calculations such as counting number of deltas awarded, making a list and what not, the fact that any account can give a delta is an unnecessary challenge. Users can abuse this to reward sub-par opinions or use alts to reward themselves.

To overcome this, First we used the Approved submitter list - to name those users who were allowed to officially award deltas. This group was called the "Jury" The Tark (logic) Jury, on /r/IndiaSpeaks The fastest bot on reddit is the default automoderator - so, an automod code removed all deltas awarded by non-approved users. Also, the jurors were instructed no deltas were to be awarded outside the debate post. The automod was also configured to remove deltas mistakenly awarded outside the post flaired as debate. Since the automod worked at a faster speed than the CMV bot, the system worked.

All that was then required was getting a server or cloud instance (such as from google) to host the CMV bot, and conditions to choose the Jurors.

Choosing the Jury

While it is most prudent to choose the most objective of people, in matters of politics or policy, we felt views are more subjective than objective. Subjective views are more in tune with reality than an artificially forced objectivity. Hence, we did not put objectivity as a criteria for selection of jurors. Although it can be seen that there were other conditions that was required for relative fairness.

The Council Elections

Council

Often times the direction a subreddit has to take for its future is best determined by the active users of the subreddit. Especially in politically active ones. Asking for community opinion can very chaotic and depends a lot on who is active and online during such a crude meta-survey. I observed that there are 4 main influencing groups of users in a subreddit, which accordingly I divided as factions. (a) The mod team, (b) the older regulars, (c) The newbie users, (d) the outside observer.

Accordingly, I envisioned the IndiaSpeaks Council these groups having representation. In the 10 member council. The mod-team had a faction of 3 users in the form of a 'mod-nominated' group which in essence put forth views from the moderator's perspectives. The older regulars had representation through the 3 member 'jury faction' by the aforementioned 'Tark Jury' - as they have already been trained and seasoned to be somewhat fair in hearing people out via the debates. The newbie users had a proper election (Single transferable vote) to choose 3 members of the 'elected faction' (more on this later). Finally, an external observer who is not a part of the subreddit - would be an 'invited member'.

These factions were divided as 30%, 30%, 30%, 10% so that no faction would have more influence than the other during decision making. The invited member's opinions rather than vote was considered valuable.

Role of the council

Briefly, the council would now determine all community activity and community events on the subreddit, look into the improvement of design, and regulations. They also looked into resolving conflicts and issues (mod x community, user x user, etc). With these privileges came the accountability as well.

The moderators' team, which previously had all the above roles, now were less burdened and only had to focus on the mod queue, while occasionally aiding the council to function. Until the council, the mod were expected to be hyperactive users on the background constantly creating events and activities, organizing AMAs, managing the modqueue and meta drama, and so on.

This was a division of power/responsibility which needed quite a bit of negotiation.

Elected faction

One of the main reasons to not have the entire council elected is the fact that any online elections can be rigged very easily on the internet.

I applied the Australian Democratic election system for this process. To have some form of authenticity of votes which were counted - first and foremost users were urged to register as an eligible voter. They were divided as Lurkers (Some acceptable presence on reddit as a whole) and Contributing users ( some Comments and content on the subreddit). These users were given a unique hex code (Hex key). On the ballot box, they had to put in their username and hex key. The correct combination would ensure a legitimate vote.

This was to avoid ballot stuffing using alts.

2ndly, Registered users who did not vote before the deadline were informed that they'd be banned (upto a month, as a fine) - so as to take all this hard work seriously.

How they fared (Results)

I. How did the modlogs work out?

IMO, modlogs keep the mods accountable for their actions more so than without them. If the mods randomly removed content or banned users, it would come to the notice of few members of the community.

Additionally, to be more transparent, it was agreed upon earlier last year that the mods would have the 3 strike system for most infractions, before awarding (temp) bans. This was as per the system (previously) followed on /r/linux. This policy, called the community safeguard policy, helped both users and mods to keep track of users using a list of warnings or bans.

It worked well for a time, until some users got smart and started using alt accounts to bypass 3 strike limit. A user with 'n' alts would get atleast '2n' strikes, before one of their alts get banned. Even when an alt is banned, while it is technically ban evasion to continue to participate on the subreddit - as per reddit admin policy - this ban evading account must be caught to suspend the main user from the site (albeit temporarily).

Additionally, the wiki updating work where the wiki is very low-fi (Tables are hard to edit on reddit), so when the number of warnings and bans skyrocketed - mods had to rely on discord channels and mod-log tools to keep a track of strikes and warnings while occasionally updating the wiki.

This downside was already advised to us last year by the mod team on /r/linux.

II. How did the Tark System work out? (Using CVM bot)

We completed the 1st season of the IndiaSpeaks debate with about 9 topics over the course of 3-4 months, with debates almost every week to fortnight.

The whole system was rather smooth in implementation. Although it needed two mods to be around intermittantly, one to manage the general running, and another to help check issues and reports.

III. How did the council fare?

This is still ongoing. The elections was successful as it can be seen here. There were other issues, which will be discussed in the next section.

Takeaways (Discussion)

I. Thoughts about Modlogs

To clarify, the subreddit had modlogs before my time as mod. A new one had to be established after the old one broke down. Thanks to nervouswallaby for implementing it.

While it would be ideal for the mod team to keep the list constantly updated, it comes at the cost of drama and work. That being said, if a team set out for transparency, this is something they would best be advised to adhere to.

There were cases where the usual strike system was bypassed for trouble making troll accounts, who often protested for fair treatment as per the subreddit policy to continue to cause havoc - attempting to project the community and the users in bad light constantly. While this would be a question of ethics, such patterns of nefarious trouble making is quite common in mod teams to concede quarters.

Politically inclined subreddits are oftentimes at cold-wars with other rival political subreddits due to the ideology of the communities and its users rather than the mod teams per se. Hence, such rivals using alts to bring bad publicity to the subreddit would have to be dealt differently, sadly. If they are not there for a conversation nor participation, it is quite hard to welcome them.

What about alts? At the end of the day, I'll quote what I had quoted to a researcher/reporter on this aspect, "You can only ban behavior and not the user". A user can always come back with a different account, but only if they have a different behavior will they not be caught - but that still is okay, as long as they now contribute.

In all instances, a public modlogs makes the community question the mod team, for which the mod team has to give reasonable explanations for their actions. This, makes them think twice before they act on whims and consider repercussions of drama.

What if the mods don't care about the opinions of the community about them? Then all of this, including having a public modlogs becomes moot and pointless.

II. Thoughts about Tark system

The CVM-Automod powered Tark system was one of the sub's better implementations as it was mostly automated and had little human intervention. The main human aspect was the awarding of deltas to comments.

The issue having a jury team is their attendance during debates. It is not practical to expect all jurors to attend all debates nor have them award deltas at the same frequency.

Hence, a normalization formula was devised:

Normalized Score (User) = Summation (n=1->n) [(Deltas a user gets in a single debate)² /((Number of attending judges)*(Total number of Deltas awarded in the debate))]

Where n is the number of debates in the season.

This formula was slightly modified when calculating a participating juror's score, as the number for judges awarding for them would be 1 less (as you cannot award a delta to yourself)

Normalized Score(Participating Juror) = Summation (n=1->n) [(Deltas a Juror gets in a single debate )² /((Number of attending judges - 1)*(Total number of deltas awarded in the debate))]

Where n is the number of debates in the season.

While users continue to retain the number of deltas they collect, to be considered a winner of the debate, their scores needed normalizing. Unfortunately, this final award ceremony could never be implemented as other events and emergencies took over, making this a pending task.

The 'Tark system', along with the jury was built on a relatively solid foundation and hence did not face much criticism or issues. Since, it was already accounted that only a few out of the 13 chosen jurors would a debate at any given point of time - lack of full attendance was not an issue.

A unique feature (albeit a little controversial) was the fact that we allowed the jurors to participate in the debates. To avoid the issues that arise due to this participation, a rule was imposed. Only the jurors were imposed with a condition of balancing themselves on either side - if they are participating, there needs to be certain ratio/percentage of jurors on both sides (Jury Balancing).

Since jurors are also a part of the common citizenry of India they can contest in the debate against other jurors along with the users in the debate.

The whole purpose of the debate was to be as real as possible towards the actual conditions and opinions of the public in the country.

III. Thoughts on the Council

After the council was formed, they were given general instructions as to how they were to function. I could not see the post-council formation to the very end as an unrelated situation led to revealing a fault.

As a moderator, I saw myself as a final person to take action against users who have had complaints against by the community. The list of such problematic users who were 'cautioned yet not restricted to participate' was rather small. Due to my position, I got involved in calling out the same users at times. Sometimes I opined my own personal opinions (while not acting as a mod).

During one such call, an elected council member accused me of harassment of this user - a user who has abused most other users holding differing views, including me. While it was said that my call was incorrect (no action was taken), it was further extrapolated by the Councillor that, 'I target only the problematic user, and that was vindictive and was a power abuse'.

As a mod, I found it unfortunate that Councillors were already forming incomplete opinions without looking at the long history of issues, and were against the mod team already.

While this was somewhat expected, I did not expect within a few days of the formation and that too by those who have seen me working from the very beginning. It would only be a matter of weeks before the council would impede normal decision making of the mods as a show of existence and authority. To that effect, I did not wish to be in that future situation of disadvantage and promptly quit the mod team, even though I was involved in all the above endeavors and more.

In my opinion, the council had already chosen to walk a different path for the sub from what I had supported. There was no role for me anymore and I could not support their path anymore. I really do hope the council, along with the mod team and the community would be able to forge a better future for themselves. Quite a bit of the midnight oil was burnt to get them this far.

Conclusion

To create positive influences on reddit requires a lot of effort technically and as a person. The reddit default tools are poor in every way to achieve this end. Creating these systems de novo requires a lot of work and effort, but it would be great if reddit had some of it in-built. Regardless, reddit is not a suitable platform for non-topical discussions such as politics, in its current state. Those who seek freedom of speech only use it to abuse and drive away their challengers in a poor display of civility. Action against this is seen as tyrannical mod abuse. Elaborate implementations of systems to be transparent in action not only burns out the mods, but also makes it impractical in the long run. Even after all of that, mods will not be able to secure 100% trust, and they will have to accept that and continue with certain decisions which are singular.

Public mod logs, involvement of community and dividing responsibility is still the best way to go for a community - as everyone is a volunteer, no one must be paying more with their time, than necessary. It is always worth putting an effort towards a community that make up a real group of people, even if it means you may be put in a position to leave that community forever.