Executing multithreaded scripts

A thread in Python is an object with a work function that is specified when an instance of the Thread class is created (import threading). An example of the implementation of the task is contained in the base script mindist.py.

The lists of processed pairs of objects are divided into parts, the number of which corresponds to the number of threads. Thread objects are created by the number of threads and then they are started by the start() method.

import threading

...

       self.numThreads = 8

       step = int(self.totalData/self.numThreads)+1

       for i in range(0, self.numThreads):

           c_from = i*step

           c_to   = i*step+step

           th = threading.Thread(target=self.CalcThread, args=(i,c_from,c_to))

           self.threads.append(th)

       for i in range(0,self.numThreads):

           self.threads[i].start()

The work function of a thread can be declared in the source code of the script once for the entire module - this does not cause a conflict between threads. A work function can also be part of a user class.

def CalcThread(self, index, c_from, c_to)

     The identifier of the created thread object must be stored (for example, in a list) - otherwise the garbage collector will work.

for i in range(0,self.numThreads):    

  …

  self.threads.append(th)

In general, it is recommended to first create all the threads, accumulating their identifiers in the list, and only then start them in a separate loop. In some cases, creating and simultaneously running a thread in the same loop can lead to undefined behavior of the script.

for i in range(0,self.numThreads):

             self.threads[i].start()    

To resolve the situation of simultaneous access of threads to the same resource (file, shared variable), you should use mutex objects.

self.mutex = threading.Lock()        

…

self.mutex.acquire()

self.processed += 1

self.mutex.release()

The lifetime of a thread depends on the complexity of the code of its work function. To prevent the script from ending before all child threads exit, you should use the join() function of the Thread class if the main thread does not control the completion of child threads itself. In this case, the main thread of the script will wait for the completion of the thread to which it is attached by the join() function. Errors that occur inside child threads can lead to their undefined behavior. Therefore , it is recommended to wrap the code of the work function of the stream in the construction try ... except ... finally.