Skip to content

Add example about std.parallelism#1760

Merged
dlang-bot merged 1 commit intodlang:masterfrom
wilzbach:add-example-parallel
Jul 20, 2017
Merged

Add example about std.parallelism#1760
dlang-bot merged 1 commit intodlang:masterfrom
wilzbach:add-example-parallel

Conversation

@wilzbach
Copy link
Contributor

image

import std.math, std.parallelism, std.stdio;

auto logs = new double[1_000_000];

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might want to initialise it to iota. Also since the point is to parallelise it maybe add sone timing code to show that it runs faster?

@CyberShadow
Copy link
Member

Eh, this is a bit boring. More interesting would be to calculate and print an actual result, such as I don't know, like sum of all prime numbers up to a point. Needs some ideas. taskPool.reduce and parallel sorting are also more interesting.

@quickfur
Copy link
Member

Maybe a benchmark to showcase the difference between parallel and non-parallel foreach.

I'm not too concerned about the example being "interesting" in the sense of doing something complicated / clever; the point of a frontpage example is to show off the language, not the cleverness of the example writer. I think the key with std.parallelism is how easy it is to turn your single-threaded loop into a parallel loop (provided, of course, that loop iterations don't depend on each other).

@quickfur
Copy link
Member

quickfur commented Jun 30, 2017

Here's a rough example that shows the difference:

void main()
{
    import std.datetime, std.math, std.parallelism, std.stdio;

    auto logs = new double[10_000_000];
    auto numIterations = 1;

    // Linear init
    auto bm1 = benchmark!({
        foreach (i, ref elem; logs)
        {
            elem = log(1.0 + i);
        }
    })(numIterations);
    writefln("Linear init: %s msecs", bm1[0].msecs);

    // Parallel init
    auto bm2 = benchmark!({
        foreach (i, ref elem; parallel(logs))
        {
            elem = log(1.0 + i);
        }
    })(numIterations);
    writefln("Parallel init: %s msecs", bm2[0].msecs);
}

@quickfur
Copy link
Member

It can probably use some trimming down to size to fit on the front page :D

@CyberShadow
Copy link
Member

I'm not too concerned about the example being "interesting" in the sense of doing something complicated / clever; the point of a frontpage example is to show off the language, not the cleverness of the example writer.

Sure, but calculating a result to memory then throwing it away is not great either.

@quickfur
Copy link
Member

OK, maybe parallel reduce might be more interesting? Like summing an array of 1,000,000 doubles.

@dlang-bot
Copy link
Contributor

Thanks for your pull request, @wilzbach!

Bugzilla references

Your PR doesn't reference any Bugzilla issue.

If your PR contains non-trivial changes, please reference a Bugzilla issue or create a manual changelog.

@wilzbach wilzbach force-pushed the add-example-parallel branch from fd6230b to 9022594 Compare July 2, 2017 00:48
@wilzbach
Copy link
Contributor Author

wilzbach commented Jul 2, 2017

It can probably use some trimming down to size to fit on the front page :D

Tried to trim it down a bit. We might want to play with the array sizes a bit.
To give you a rough feeling:

500_000 x 100:
Linear init: 1639 msecs
Parallel init: 655 msecs

300_000 x 100:
Linear init: 988 msecs
Parallel init: 341 msecs

100_000 x 100:
Linear init: 313 msecs
Parallel init: 118 msecs

So the performance of the DLang-Tour backend, isn't that bad ;-)

Run it yourself.

Sure, but calculating a result to memory then throwing it away is not great either.

Hmm this (iteration of a list in parallel) is actually my most common use of std.parallelism.
The main point to show here is how amazingly easy it is to do parallelism in D.

@wilzbach wilzbach force-pushed the add-example-parallel branch from 9022594 to 0c17c3d Compare July 10, 2017 23:12
@wilzbach wilzbach force-pushed the add-example-parallel branch from 0c17c3d to 213e1c5 Compare July 19, 2017 18:56
@wilzbach
Copy link
Contributor Author

So ... imho just showing off how easy & beautiful parallel algorithms can be is a major selling point (and we now easy benchmark to prove the point).

@CyberShadow what do you think?

@CyberShadow
Copy link
Member

Alright.

Lines wrap at the default size:

@wilzbach
Copy link
Contributor Author

Interesting - you seem to use a different font on your FF:

image

@CyberShadow
Copy link
Member

Oh yeah, monospace can mean anything I guess. Okay.

@CyberShadow
Copy link
Member

on your FF

Not browser specific BTW

@wilzbach
Copy link
Contributor Author

monospace can mean anything I guess. Okay.

Btw:

font-family: Consolas, "Bitstream Vera Sans Mono", "Andale Mono", Monaco, "DejaVu Sans Mono", "Lucida Console", monospace;

I guess you don't have Consolas installed?

@wilzbach
Copy link
Contributor Author

Interesting that the bot doesn't merge this ... will investigate.

@dlang-bot dlang-bot merged commit 386cda7 into dlang:master Jul 20, 2017
@wilzbach wilzbach deleted the add-example-parallel branch July 20, 2017 00:54
@wilzbach
Copy link
Contributor Author

I saw the following in the logs:

» 20 Jul 2017 02:50:40.247  163 <190>1 2017-07-20T00:50:39.742281+00:00 app web.1 - - [A3313FA1:00000000 dbg] Connection was closed by remote peer (fd 12).
» 20 Jul 2017 02:50:40.247  196 <190>1 2017-07-20T00:50:39.742322+00:00 app web.1 - - [A3313FA1:9C3A0DA1 dbg] Client didn't send the initial request in a timely manner. Closing connection.

...

 20 Jul 2017 02:52:42.330  137 <190>1 2017-07-20T00:52:41.815660+00:00 app web.1 - - [A3313FA1:9C3A0DA1 dbg] start task (fd 12).
» 20 Jul 2017 02:52:42.330  124 <190>1 2017-07-20T00:52:41.815686+00:00 app web.1 - - [A3313FA1:00000000 dbg] FD: -1
» 20 Jul 2017 02:52:42.330  164 <190>1 2017-07-20T00:52:41.815735+00:00 app web.1 - - [A3313FA1:00000000 dbg] Socket event on fd 12: 17 (33FCA88 vs 33FCA88)
» 20 Jul 2017 02:52:42.330  163 <190>1 2017-07-20T00:52:41.815770+00:00 app web.1 - - [A3313FA1:00000000 dbg] Connection was closed by remote peer (fd 12).
» 20 Jul 2017 02:52:42.330  196 <190>1 2017-07-20T00:52:41.815808+00:00 app web.1 - - [A3313FA1:9C3A0DA1 dbg] Client didn't send the initial request in a timely manner. Closing connection.
» 20 Jul 2017 02:52:42.330  135 <190>1 2017-07-20T00:52:41.815811+00:00 app web.1 - - [A3313FA1:9C3A0DA1 dbg] task out (fd 12).
» 20 Jul 2017 02:52:42.330  132 <190>1 2017-07-20T00:52:41.815817+00:00 app web.1 - - [A3313FA1:9C3A0DA1 dbg] task finished.

Will upgrade the bot to Vibe 0.8.0 later -maybe that helps.

@CyberShadow
Copy link
Member

I guess you don't have Consolas installed?

That's a Microsoft font and IIRC not a core web font so that's probably it.

@wilzbach
Copy link
Contributor Author

Will upgrade the bot to Vibe 0.8.0 later -maybe that helps.

FYI: dlang/dlang-bot#137 (deployed the upgrade of the bot to Vibe 0.8.0)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants