Remove odd disqus comments

[wilzbach]

Disqus has now been tried out for more than two years, and imho didn't work well.
The only comments we got are like those that should go to the forum:

You can see the other comments here:
https://disqus.com/home/forum/vibe-d/

There's also a discussion two years ago I found on the forum: http://forum.dlang.org/thread/lr94p4$spu$1@digitalmars.com

The best idea was to integrate stack overflow into the site and imho that's a great idea and a lot more helpful than disqus comments.

Ping @s-ludwig @Dicebot @jmdavis

[s-ludwig]

It should be noted that the /library/ pages were quite hidden and my gut feeling is that the vast majority only knew or used the /phobos/ ones. So I'd take the current amount of comments with a grain of salt. But one thing is clear, we need a few people who feel responsible for replying or moderating comments.

[JackStouffer]

we need a few people who feel responsible for replying or moderating comments.

And I don't think that's reasonable. IMO, let's keep the technical discussion in the n.g. and on SO. The less places to moderate the better.

[mihails-strasuns]

I was always in favor of "Create new question in forum/NG" button instead.

[CyberShadow]

Yep, what I was thinking of is to embed a forum widget, which would present a NG thread as a comment thread. Haven't gotten around to implementing it, though.

[s-ludwig]

Integrating this with the forum definitely has a lot of merit, but while it will in all likelihood increase participation, I don't think that it will remove the need for moderators. Listing a huge forum thread within the documentation doesn't make sense, as the comments are meant as an additional reference, but such threads will surely happen. A voting functionality also sounds like a useful feature to enable basic crowd-sourced moderation.

[CyberShadow]

Good point. It will call for a new view mode anyway. Perhaps it should only show top-level comments and collapse replies by default. I think the highest signal/noise will be in the top-level comments, as they will be addressing the documented symbol directly.

[mihails-strasuns]

I was not thinking of embedding but of a shortcut to create new topic in D.learn with title akin to [std.algorithm.iteration] Question.

[CyberShadow]

Could've sworn that was implemented already.

Well, it is now.

CyberShadow/DFeed@c3199d6

http://forum.dlang.org/newpost/learn?subject=[std.algorithm.iteration]%20Question

[andralex]

FWIW I am in favor of keeping disqus. I don't think there's enough evidence to discard it. It lowers the barrier of entry to beginners and it creates the opportunity to add value to the documentation with ease.

But delegation of responsibility is what it is. Vladimir is the Web everything czar, so I am deferring to him.

[CyberShadow]

OK, then I suggest to shelve this until we have a concrete proposal for a replacement, which is probably something I'll need to get around to doing someday.

[dnadlinger]

@andralex:

I don't think there's enough evidence to discard it.

I don't think there is any evidence that supports keeping it. It appears like there has been a grand total of 10 discussions over the course of three years, with no discussions showing for the past year.

Plus, the system didn't exactly work out great in most of those instances, although that could be admittedly be improved by persuading some of the regulars to monitor the Disqus forum.

[MartinNowak]

At least it supports voting for comments.
But compared to using our very active forum which can be used anonymously, creating an account with disqus takes at least 2 steps and has quite some fine print attached.
Terms of Service
Privacy Policy

[MartinNowak]

Stay or go?

[CyberShadow]

I am mildly in favor of 'stay' until we have a replacement, mainly because it doesn't require changing anything and it's not costing/hurting us actively to keep it.

[JackStouffer]

Go

• Slows down page load times for little gain
• Is not privacy focused

[CyberShadow]

@JackStouffer Good points. Feel free to remove it then.

[JackStouffer]

Good points. Feel free to remove it then.

???

I don't have merge rights.

[CyberShadow]

Ah right. Forgot I was commenting on a PR to do exactly this. Thanks Martin.

[MartinNowak]

The idea of a small widget to post to the forum remains appealing.

[andralex]

I just took a look at the dlang.org use statistics and it seems ddox has had very low use.

In July 2016 (most recent complete month), the most visited page containing "/library" has been http://www.dlang.org/library itself, totaling only 435 hits. Next down the list is http://www.dlang.org/library/std/range/ref_range.op_assign.html with 340 hits. There's no long tail, in fact there's only one more (excluding .js paraphernalia) in the top 256: http://www.dlang.org/library-prerelease/ with 231 hits. For contrast, the top page with "/phobos" in its name is http://www.dlang.org/phobos with 7351 hits follow by a long tail of about 80 pages among the top 256.

So one step we need to take is increase the visibility of /library before drawing any conclusion about popularity of features on it.

Regarding the ideological argument, I don't have time and energy for yet another debate but I will say this: it's the internet, get used to it. Whether you search, browse, or use whatever services, the creators thereof are in it to be paid. I don't think disqus is any worse than anyone else, and as long as we don't have a replacement it's a good thing to have.

[andralex]

@MartinNowak could you please unmerge this. Thanks. I'd appreciate it if I'd be kept in the loop in policy decisions.

I'll send you and Vladimir the password for looking at the site stats.

[adamdruppe]

On Tue, Aug 23, 2016 at 07:27:53AM -0700, Andrei Alexandrescu wrote:

@MartinNowak could you please unmerge this. Thanks. I'd appreciate it if I'd be kept in the loop in policy decisions.

At some point, you're going to have to allow yourself to delegate.

[CyberShadow]

In July 2016 (most recent complete month), the most visited page containing "/library" has been http://www.dlang.org/library itself, totaling only 435 hits. Next down the list is http://www.dlang.org/library/std/range/ref_range.op_assign.html with 340 hits. There's no long tail, in fact there's only one more (excluding .js paraphernalia) in the top 256: http://www.dlang.org/library-prerelease/ with 231 hits. For contrast, the top page with "/phobos" in its name is http://www.dlang.org/phobos with 7351 hits follow by a long tail of about 80 pages among the top 256.

I think you need to sum up all the hits under the /phobos/ and /library/ prefixes to get a full picture. DDox gets good visibility from search engines, as it is better SEO-friendly when searching for a particular symbol.

it's the internet, get used to it.

I don't want to waste time arguing on this either but I'll just say that I think many will disagree.

[andralex]

@CyberShadow: I just sent you the password, look at the data, draw your own conclusions, and post them here.

So we have two arguments:

• Not enough use of disqus
• Disqus is bad because privacy etc.

@CyberShadow and @MartinNowak can see how the first argument holds. I think the data blows it away out of the water. The second is a topic of debate in which reasonable people may disagree. So someone must make a judgment call. I'm uncomfortable with such a judgment call being pushed on me as the only reasonable thing to do.

[CyberShadow]

There's also:

we need a few people who feel responsible for replying or moderating comments.

[andralex]

@CyberShadow I'm already a moderator, and that's going to be a problem good to have. Looking forward to having a bunch of content to moderate.

Posting about a library page on the forum is a completely different flow than having user-produced content on the page itself. I don't see how one obviates the other.

I don't have a particular opinion about disqus (other that I see it on reputable sites) but I do like the functionality it provides. If @CyberShadow could provide the same for free, that would be awesome. But going around with "Kill disqus!" doesn't sound like a productive way to go about things.

[CyberShadow]

@CyberShadow: I just sent you the password,

Thanks but I didn't need it.

look at the data, draw your own conclusions, and post them here.

vladimir@digitalmars:/home/vladimir % zgrep "GET /phobos/" /var/log/httpd/dlang.org/httpd-access.log* | wc -l
   17918
vladimir@digitalmars:/home/vladimir % zgrep "GET /library/" /var/log/httpd/dlang.org/httpd-access.log* | wc -l
   33545

I'm afraid that the real numbers look very different, due to my point above.

[MartinNowak]

Setting up Isso for self-hosted comments is really trivial, i.e. apt-get install isso.
Example Reducing vibe.d turnaround time (Part 1 Faster Linking).

[MartinNowak]

But the main question is whether we'll actually be able to maintain the comments.

[andralex]

@MartinNowak: again, that'll be a good problem to have. Per the Romanian joke: Romania doesn't invade China because they don't have room for all the prisoners.

[MartinNowak]

Feel free to undo the merge, running out of battery here, close to China btw.

[andralex]

@CyberShadow please look closer. Most GETs fetch http://www.dlang.org/library/symbols.js.

[andralex]

@MartinNowak Have fun traveling. Best time to deploy major changes eh :). @CyberShadow agree to undo until @MartinNowak returns and we can have a close look at things?

[CyberShadow]

@CyberShadow please look closer. Most GETs fetch http://www.dlang.org/library/symbols.js.

vladimir@digitalmars:/home/vladimir % zgrep "GET /phobos/.*html" /var/log/httpd/dlang.org/httpd-access.log* | wc -l
   16627
vladimir@digitalmars:/home/vladimir % zgrep "GET /library/.*html" /var/log/httpd/dlang.org/httpd-access.log* | wc -l
   32230
vladimir@digitalmars:/home/vladimir % zgrep "GET /library/symbols.js" /var/log/httpd/dlang.org/httpd-access.log* | wc -l
    1215

This is not a definition of "most" that I am familiar with :)

[andralex]

@CyberShadow How do you explain the discrepancy with webalizer?

[andralex]

@MartinNowak @CyberShadow I'll do something in premiere: will leave this entirely up to you. You know what I think and it is my personal opinion that the entire ideological argument is crap, but you decide on this.

Feels terrible but as @adamdruppe said at some point I got to delegate stuff.

[andralex]

@CyberShadow still curious about the log vs webalizer thingamaboob.

[CyberShadow]

@CyberShadow How do you explain the discrepancy with webalizer?

I'm not familiar with this software. I know it works by reading the same logs I grep above, so it's either buggy or somehow selective in its processing. It keeps a tally and archive for longer periods than the web server logs, so the scale of the numbers will not match, however I don't know what causes the mismatched proportions.

Feel free to undo the merge, running out of battery here, close to China btw.

I don't really have a strong opinion one way or the other but it seems to me that the arguments have been piling up in favor of "kill" and evaporating away from the favor of "keep". Feel free to undo if it means we can stop wasting time on this debate, though.

[s-ludwig]

It's definitely necessary to filter out search engine bots from the logs. Since /library/ has a lot more pages than /phobos/, that makes a huge difference.

[CyberShadow]

It's definitely necessary to filter out search engine bots from the logs. Since /library/ has a lot more pages than /phobos/, that makes a huge difference.

I thought that could be it, but the logs do not mention user agents at all... so Webalizer wouldn't be able to filter them out either.

[s-ludwig]

BTW, I'd generally also be in favor of replacing it with something else instead of just removing it now. It's possible to post without registration, so it isn't that bad privacy wise. And remember that everything is public anyway, just like the forum/newsgroup.

[CyberShadow]

It's possible to post without registration, so it isn't that bad privacy wise. And remember that everything is public anyway, just like the forum/newsgroup.

I think the major privacy concern is that Disqus can track visitors along all websites that use Disqus, whether they're signed up or not, and whether they participate or not.

[CyberShadow]

Here is a more accurate measure - counting unique IPs:

vladimir@digitalmars:/home/vladimir % zgrep "GET /phobos/" /var/log/httpd/dlang.org/httpd-access.log* | cut -d ' ' -f 1 | sort -u | wc -l
    5649
vladimir@digitalmars:/home/vladimir % zgrep "GET /library/" /var/log/httpd/dlang.org/httpd-access.log* | cut -d ' ' -f 1 | sort -u | wc -l
    2332

This should better represent the usage share, if we make the assumption that major search engines visited both parts of the website from most of their pool of IPs.

[CyberShadow]

@CyberShadow How do you explain the discrepancy with webalizer?

I think I know:

1. The discrepancy with symbols.js is simply due to the different time frame. Webalizer shows stats one month at a time, whereas the web server's logrotate only keeps logs for the past 10 days. Indeed, 3265 (Webalizer hits for August for symbols.js) /23*10 =~ 1419, which is in the ballpark of my 1215.
2. The discrepancy between /phobos/ and /library/ is due to the long tail which Webalizer doesn't show. Too bad we don't log referrers, would be interesting to know how people reach these pages (site navigation vs. search engines)... I guess it's still possible by looking at past requests from the same IP, but not trivial.

[CyberShadow]

is due to the long tail which Webalizer doesn't show

Wait, just saw the "Show all URLs" link. Well, Webalizer seems to agree, actually:

vladimir@digitalmars:/home/vladimir % cat /usr/local/www/dlang.org/data/Usage/HTTP/url_201608.html | grep '  /phobos/' | cut -d ' ' -f 1 | awk '{s+=$1} END {print s}' 
39599
vladimir@digitalmars:/home/vladimir % cat /usr/local/www/dlang.org/data/Usage/HTTP/url_201608.html | grep '  /library/' | cut -d ' ' -f 1 | awk '{s+=$1} END {print s}'
83353

OK, that's probably enough numbers :)

[andralex]

@CyberShadow thanks for the analysis!

[s-ludwig]

I think the major privacy concern is that Disqus can track visitors along all websites that use Disqus, whether they're signed up or not, and whether they participate or not.

Okay, fair point. Not trying to justify/excuse that, but it should be noted that the same is more or less true for the Twitter widget and the jquery and font-awesome includes. Good that the Google ad isn't there anymore.