multithreading – Improved Performance When Parallel Python Object Execution

I have a high performance application requiring an intensive compute that runs on a Centos 7 machine with 2.7.5.

I will try to explain what the application does in almost any detail

  1. The application runs an infinite loop, where it receives a message from an API call representing the power levels of a device. The message is in Avro and encoded as a JSON string.
  2. Each device gets a maximum of 8 separate power level readings (imagine that each of them is an independent hardware component in the device). 8 separate components make up one device. There are 50 in total. So, power level ratios (50 * 8) are possible
  3. Each of these 8 hardware devices report a power report once every 30 seconds.
  4. I have a business logic to calculate an arithmetic mean of the first 4 devices (component ID 1-4) and an average for the last 4 devices (ID 5-8)
  5. For a given device, once the 8 received readings received, I calculate the average above and compare the difference between the average of the group and the individual components, that is to say the first 4 – means_1, the last 4 – means_2.

    for id in 1 2 3 4: do (mean_1 - pwr_reading (id))
    for id in 5 6 7 8: do (mean_2 - pwr_reading (id))
    
  6. If the difference above is below a certain threshold, say thresh_first for the first four and thresh_last for the last four, I have to do an action.


So, to model the requirements above, I've created a device class in which I hold this information.

obj_list = {}

DevPwrInfo class (object):
"" "The class provides an abstraction of all the treatments we do at the same time
level of the device
"" "

def __init __ (auto, code):
"" "The constructor launches a new device object by initializing identifiers
and the data structures needed for the evaluation
"" "
self.code = code
self.first4_pwr = {}
self.last4_pwr = {}
self.mean_val_first4 = ""
self.mean_val_last4 = ""
self.threshold_breach_list_first4 = []
        self.threshold_breach_list_last4 = []


    def reset_dev_info (auto):
"" "Clears retained data after completing a series of reports
Evaluation
"" "
self.first4_pwr = {}
self.last4_pwr = {}
self.mean_val_first4 = ""
self.mean_val_last4 = ""

def add_dev_pwr (auto, id, pwr, pwr_valid_flag):
if 1 <= int (id) <= 4:
if pwr_valid_flag:
self.first4_pwrhttps://codereview.stackexchange.com/q/212261 = pwr
other:
self.first4_pwrhttps://codereview.stackexchange.com/q/212261 = 0.0
other:
if pwr_valid_flag:
self.last4_pwrhttps://codereview.stackexchange.com/q/212261 = pwr
other:
self.last4_pwrhttps://codereview.stackexchange.com/q/212261 = 0.0

if len (self.first4_pwr) == 4:
self.mean_val_first4 = self.compute_mean (first4_pwr)
self.compare_thresh (& # 39; first4 & # 39;)
self.first4_pwr.clear ()

if len (self.last4_pwr) == 4:
self.last4_pwr = self.compute_mean (last4_pwr)
self.compare_thresh (& # 39; last4 & # 39;)
self.last4_pwr.clear ()

def compute_mean (self, pwr_list):
return (float (sum (pwr_list)) / max (len (pwr_list), 1))

def compare_thresh (self, type):
low_thresh = thresh_dict[self.code]

        if you type == & # 39; first4 & # 39 ;:
pwr_dict = self.first4_pwr
mean_val = self.mean_val_first4
other:
pwr_dict = self.last4_pwr
mean_val = self.mean_val_last4

for id, pwr in pwr_dict.iteritems ():
if int (math.floor (mean_val - (pwr))) <int (low_thresh):
print ("will add more logic here")

def pwr_report_msg_decode (message):
"" "Manager for API raw idb message
"" "
if the message is not None:
# This API is called for each message from the API call, so that
# The object of each device is called by the string identifier and
# The function & # 39; add_dev_pwr & # 39; will ensure that the lists are up-to-date
obj_list[message['code']].add_dev_pwr (message['id'],
message['pwr'],
message['valid_flag'])

# obj_dict is a dictionary of objects with the key name as device name as value as
# dict object
if __name__ == "__main__":
# allowed_devices_list contains the list of 44 device names
allowed_devices_list = [ 'abc01', 'def01', 'xyz01' ]
    for device in allowed_devices_list:
obj_list[device] = DevPwrInfo (device)

in truth:
# An API producing a message in the format
msg = {"code": "abc01", "id": "3", "pwr": "-59.2", "valid_flag": "True & # 39;"}

So my question is how to make sure that each of the 44 objects runs in parallel and not sequentially in a thread. I watched ThreadPoolExecutor but you do not know how to make it optimum in terms of calculation?