I have a program on Windows that tries to distribute its work over multiple threads. They key function is this:

proc doJobs(config: Config) =
  var responses = newSeq[FlowVarBase]()
  for filename in getFilenames(config):
        responses.add(spawn doOneJob(config, filename))
  sync()

It never uses more than 1 CPU (i.e., the load is <=25% and there are 4 CPUs). I have the line threads:on in nim.cfg in the same directory as the .nim file, and now I call setMinPoolSize(countProcessors() * 2) but to no avail.

Also, I can't see how to read the data from the responses (doOneJob returns a string). I tried this approach:

proc doJobs(config: Config) =
  var responses = newSeq[FlowVarBase]()
  for filename in getFilenames(config):
    responses.add(spawn doOneJob(config, filename))
  sync()
  while responses.len() > 0:
    let i = awaitAny(responses)
    # HOW to retrieve the result string from responses[i] ???
    responses.delete(i)
But I don't know how to retrieve each result.

(It used to crash but that was my bad, now fixed.)

2017-03-17 14:33:29
Maybe doOneJob needs to be significantly more expensive for the threading to kick in? If it finishes too quickly, you won't see more than one core busy.
2017-03-17 19:59:03

doOneJob is very expensive -- and it now peaks at over 40% CPU, but I can't get it to do more than that, at least not so far.

And I also don't know how to get the string out of each response -- I'd really like to be able to do that. (Right now I'm having each thread output its own data but that means having a lock to protect the echo calls.)

2017-03-17 22:17:34

Well FlowVarBase is the base class, type it properly and then you can get the result back via ^:

import strutils, math, threadpool

proc term(k: float): float = 4 * math.pow(-1, k) / (2*k + 1)

proc computePI(n: int): float =
  var ch = newSeq[FlowVar[float]](n+1)
  for k in 0..n:
    ch[k] = spawn term(float(k))
  for k in 0..n:
    result += ^ch[k]

echo formatFloat(computePI(5000))

2017-03-18 07:32:45
Oh and config is deepcopied into the thread, that can be very expensive, better pass to doOneJob only what it needs or use a shared hash table and have the threads use shared memory.
2017-03-18 07:40:06