@Mikra, Depending on the compiler, unspecified usually means asking for trouble and lots of sleepless nights! lol

With the above code, I lose both signals, and always get a deadlock, even with a single wait.

I believe this is because the signals happen 5000ms before the main thread waits on the condition, and the signals are not saved.

If I reverse the roles of the threads, like the code below, everything works, no deadlocks.

But, if I un-comment the second wait, I deadlock, the second signal is lost. (Again, I expected this to happen.)

If I then place a sleep between the two signals in the following code, no deadlock, no lost signals.

All of these results enforce my understanding that signals are never saved.

I don't have a windows box atm, but I will in a few days, and I will test it there. Maybe Microsoft did weird stuff with their implementation of condition variables?

import locks,os

# globals
var
  thr: Thread[int] # don´t know if there is a untyped thread possible
  cLock: Lock
  lockCond: Cond

proc threadFunc (param : int) {.thread.} =
  echo "start waiting"
  withLock(cLock):
    wait(lockCond,cLock)
    #wait(lockCond,cLock)
# uncomment to check if there is a queue behind; on windows the (expected)
# behaviour is a deadlock for the second wait (second signal call is lost)
  echo "end waiting"

initLock(cLock)
initCond(lockCond)

createThread(thr, threadFunc,0)

echo "mainthread:begin"
sleep(5000) # ensure that the signal of the childthread is called first

signal(lockCond) # first signal call
signal(lockCond) # if you signal twice the second signal is lost (no queue)
echo "mainthread: signal executed"

joinThreads(thr) # should not block if the childthread is already finished
deinitLock(cLock)
deinitCond(lockCond)
echo "end"

2017-10-08 21:41:28

I remember writing a consumer pattern using 2 threads for consumer and producer. Using channel to send the message, and using cond to wait if there's nothing to consume yet.

import locks
from strutils import parseInt
from os import sleep

when not defined(vcc):
  import threadpool

var
  chan: Channel[int]
  cond: Cond
  lock: Lock
  time = 700

initLock lock
initCond cond
chan.open

proc chanWaiting() {.thread.} =
  while true:
    var res = tryRecv chan
    if res.dataAvailable:
      echo "Get the num: ", res.msg
      
      # to simulate the congestion or slow operation
      sleep time
      
      if res.msg > 10:
        echo "stopping operation"
        break
      
      # immediately for next loop if there's a value to avoid message
      # congestion, in the event of there's no message then it will wait
      # for the condition signaled
      continue
    
    echo "waiting for next iteration"
    cond.wait lock
    echo "condition signaled"

proc chanSending() {.thread.} =
  while true:
    var num = try: stdin.readline.parseInt
              except: -1
    echo "sending ", num
    chan.send num
    echo "going to signal the condition"
    cond.signal
    if num > 10:
      break

when defined(vcc):
  var
    threadsend: Thread[void]
    threadwait: Thread[void]
  
  threadsend.createThread chanSending
  threadwait.createThread chanWaiting
  
  joinThread threadsend
  joinThread threadwait
else:
  spawn chanWaiting()
  spawn chanSending()
  sync()

From what I remembered, signaling condition before the condition waiting will make the signal lost. Also, signaling no matter how many are done, condition will only be handled once.

2017-10-09 18:43:00

@mashingan, interesting example. The consumer pattern is the classic use case for a condition variable.

From what I remembered, signaling condition before the condition waiting will make the signal lost.

We agree on this point. The signal will be lost if you signal before the wait.

Also, signaling no matter how many are done, condition will only be handled once.

this is not correct. You only observe this because you only have one consumer thread.

If you had multiple consumer threads waiting on the condition, then each thread would need to be signaled.

The signal would not be "handled only once" as you say. It would be handled once for each waiting thread. (Or a broadcast call would need to be made to wake all waiting threads at once).

The canonical use case for a condition variable is single producer, multiple consumer. So multiple signals must be handled.


One other small thing is that it looks like your sample code has the same bug as @mikra, you do not acquire the lock before waiting on the condition in your chanWaiting() thread.

This may or may not affect anything, it depends on the underlying OS implementation of condition variable, but it's a good idea in order to prevent weird behavior (such as lost signals).

2017-10-09 22:41:52

@rayman2220 & @mashingan many thanks for your inputs. The locking/signalling stuff seems a little bit nasty because of the behaviour of the underlying layer (OS); like always the details hurt

For some usecases (especially you are looping withing the producer/consumer) it doesn`t matter if a signal is lost (signalling before owning and waiting on the lock). @mashingan besides of @rayman2220´s comments on your code I think it could be possible that (under some circumstances) your last signal-call is lost (n=10) so the chanWaiting thread could possible deadlock.

For my usecase I have a "oneshot"-like behaviour because the thread is calling signal() only once. So it´s extremely nasty in my eyes that the signal could be lost. I thought about that and finally I come up with the following solution now (testcode below). Please correct me if I missed something and the example is deadlocking on your environment (for me on windows it´s working @rayman2220:no clue how microsoft has implemented that but it seems that the signal-call on a condition is acting like a latch i.e. at least one signal-call is preserved):

import locks,os

# globals
var
  thr: Thread[void]
  cLock: Lock
  lockCond: Cond
  rcvWaiting : bool = false

proc threadFunc () {.thread.} =
  echo "childthread: starting up"
  while true:
  # we like to model a one-shot behaviour so ensure that the receiver is owning the lock
    if atomicLoadN[bool](unsafeAddr[bool](rcvWaiting),ATOMIC_SEQ_CST):
      signal(lockCond)
      echo "childthread: signal sent"
      break
    sleep(200) # ugly
  
  echo "childthread: terminated"

initLock(cLock)
initCond(lockCond)

createThread(thr, threadFunc)

echo "mainthread:begin"
sleep(3000) # simulate childthread is running before we enter the lock
echo "mainthread: start waiting"

withLock(cLock):
  # signal the sender that we are owning the lock now
  atomicStoreN[bool](unsafeAddr[bool](rcvWaiting),true,ATOMIC_SEQ_CST)
  wait(lockCond,cLock)

echo "mainthread: end waiting, sig received"
joinThreads(thr)
deinitLock(cLock)
deinitCond(lockCond)
echo "mainthread:end"

2017-10-09 23:31:24

@Mikra,

Your code works on my machine, and seems sound. I think I have a cleaner solution though.

How about this:

import locks,os

# globals
var
  thr: Thread[void]
  cLock: Lock
  lockCond: Cond

proc threadFunc () {.thread.} =
  echo "childthread: starting up"
  # one shot behavior can be achieved by leveraging the lock on the sender side as well
  withLock(cLock):
    signal(lockCond)
    echo "childthread: signal sent"
  
  echo "childthread: terminated"

initLock(cLock)
initCond(lockCond)

echo "mainthread:begin"
echo "mainthread: start waiting"

withLock(cLock):
  # Create the sending thread after we have alread acquired the lock.
  # The sender thread will be forced to wait for us because we own the lock.
  createThread(thr, threadFunc)
  # The wait function atomically releases the lock, allowing the sender thread to continue.
  wait(lockCond,cLock)

echo "mainthread: end waiting, sig received"
joinThreads(thr)
deinitLock(cLock)
deinitCond(lockCond)
echo "mainthread:end"

Look! no timeouts!

Create the thread after you have already acquired the lock. Then you can leverage the lock on both the sender and receiver end, along with the property that waiting on a condition variable atomically unlocks the lock. The receiver thread will atomically re-acquire the lock as soon as it wakes up again.

This gives you a "latch" or "synchronization barrier" to hook in to. (Pick your terminology based your favorite text-book on multi-threading lol)

2017-10-10 18:52:01

@rayman2220:no clue how microsoft has implemented that but it seems that the signal-call on a condition is acting like a latch i.e. at least one signal-call is preserved

damnit Microsoft..... y u have to be so broken?! lol

2017-10-10 18:55:23

@rayman22201 your solution is much more elegant than mine and works also perfectly ; many thanks for that. yes the wait has to release the lock - anything else makes no sense (unfortunately send_all is missing in the locks-module). Decades ago I did hardware-design, so the latch is my mental model

At the moment I am about to implement a softtimer-pool solution in Nim with a locking (wait) and a nonblocking (polling) api. I am not sure if I can afford acquiring a lock in my worker-thread because I like a one-thread solution and the "wait_for" stuff is missing in Nim (at the moment I am still at the beginner-street there so it could be possible I have to go with my clumsy solution. When finished I will push it on github. But anyway many thanks for your great support

2017-10-10 21:48:36

Glad I could help!

I don't want to disrespect your solution! You have basically implemented a spin lock, which is very high performance.

As long as you can afford to burn cpu cycles on the wait loop inside the worker, it is a good solution.

Like all important engineering problems, it is a design trade-off.

I look forward to seeing your library when it's published

2017-10-11 17:13:07
@mikra, are you not considering asyncdispatch module?
2017-10-11 17:50:39

@rayman22201 for my very specific case (softtimerpool) the tickthread needs to sleep (after doing the counting-,signalling- and api-handling-stuff) to obtain the user-defined timebase for the artificial ticks and I don´t like a thread for each counter (that would be resource-burning . My solution should also run within resource-limited environments (tinyavr for instance).

@mashingan yes thanks for your hint; I looked at it but thats not multi-threaded. It runs inside the same thread(polling) - it`s for asynchronous I/O. I need a own thread to obtain the artificial ticks. Its not for critical timing purposes; often protocols need that (tcp/ip for instance) or for dimming LED´s and so on.

2017-10-11 19:03:39
<<<••12••>>>