[SOLVED] Python: pixel manipulation pefrormance. Virtual desktop for embedded device

Issue

I am looking for an efficient way of pixel manipulation in python.
The goal is to make a python script that acts as virtual desktop for embedded system.
I already have one version that works, but it takes more than a second to display single frame (too long).

Refreshing display 5 times per second would be great.

How it works:

  1. There is an electronic device with microcontroller and display (128x64px, black and white pixels).
  2. There is a PC connected to it via RS-485.
  3. There is a data buffer in microcontroller, that represents every single pixel. Lets call it diplay_buffer.
  4. Python script on PC downloads diplay_buffer from microcontroller.
  5. Python script creates image according to data from diplay_buffer. (THIS I NEED TO OPTIMIZE)

diplay_buffer is an array of 1024 bytes. Microcontroller prepares it and then displays its content on the real display. I need to display a virtual copy of real display on PC screen using python script.

How it is displayed:

Single bit in diplay_buffer represents single pixel.
display has 128×64 pixels. Each byte from diplay_buffer represents 8 pixels in vertical. First 128 bytes represent first row of pixels (there is 64px / 8 pixels in byte = 8 rows).

I use python TK and function img.put() to insert pixels. I insert black pixel if bit is 1 and white if bit is 0. It is very ineffective.
Meybe there is diffrent class than PhotoImage, with better pixel capability?

I attach minimum code with sample diplay_buffer. When you run the script, you will see the frame and execution time.

Meybe there would be somebody so helpful to try optimize it?
Could you tell me faster way of displaying pixels, please?

denderdale

Sample frame downloaded from uC

And the code (you can easily run it)


#this script displays value from uC display buffer in a python screen
from tkinter import Tk, Canvas, PhotoImage, mainloop
from math import sin
import time

WIDTH, HEIGHT = 128, 64
ROWS = 8

#some code from tutorial... check what it does:
window = Tk()
canvas = Canvas(window, width=WIDTH, height=HEIGHT, bg="#ffffff")
canvas.pack()
img = PhotoImage(width=WIDTH, height=HEIGHT)
canvas.create_image((WIDTH/2, HEIGHT/2), image=img, state="normal")


#this is sample screen from uC. It is normally periodically read from uC on runtime to refresh screen view. 
diplay_buffer =bytes([16, 16, 16, 16, 16, 0, 16, 16, 16, 16, 16, 0, 16, 16, 16, 16, 16, 0, 16, 16, 16, 16, 16, 0, 16, 16, 16, 16, 16, 0, 16, 16, 16, 16, 16, 0, 16, 16, 16, 16, 16, 0, 0, 0, 0, 0, 0, 0, 0, 130, 254, 130, 0, 0, 254, 32, 16, 8, 254, 0, 254, 144, 144, 144, 128, 0, 124, 130, 130, 130, 124, 0, 0, 0, 0, 0, 0, 0, 16, 16, 16, 16, 16, 0, 16, 16, 16, 16, 16, 0, 16, 16, 16, 16, 16, 0, 16, 16, 16, 16, 16, 0, 16, 16, 16, 16, 16, 0, 16, 16, 16, 16, 16, 0, 16, 16, 16, 16, 16, 0, 16, 16, 16, 16, 16, 0, 0, 0, 18, 42, 42, 42, 36, 0, 28, 34, 34, 34, 28, 0, 0, 16, 126, 144, 64, 0, 32, 32, 252, 34, 36, 0, 0, 0, 40, 0, 0, 0, 0, 0, 0, 0, 0, 0, 4, 2, 130, 252, 128, 0, 4, 42, 42, 30, 2, 0, 62, 16, 32, 32, 30, 0, 0, 0, 0, 0, 0, 0, 0, 66, 254, 2, 0, 0, 130, 132, 136, 144, 224, 0, 0, 0, 0, 0, 0, 0, 78, 146, 146, 146, 98, 0, 124, 138, 146, 162, 124, 0, 78, 146, 146, 146, 98, 0, 78, 146, 146, 146, 98, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 15, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 15, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 255, 0, 0, 0, 0, 0, 0, 254, 16, 16, 16, 254, 0, 28, 42, 42, 42, 24, 0, 0, 130, 254, 2, 0, 0, 0, 130, 254, 2, 0, 0, 28, 34, 34, 34, 28, 0, 0, 0, 0, 0, 0, 0, 254, 144, 144, 144, 128, 0, 62, 16, 32, 32, 16, 0, 0, 34, 190, 2, 0, 0, 28, 42, 42, 42, 24, 0, 62, 16, 32, 32, 30, 0, 28, 34, 34, 20, 254, 0, 0, 0, 250, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 255, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 255, 0, 0, 0, 0, 0, 0, 124, 130, 130, 130, 68, 0, 4, 42, 42, 30, 2, 0, 62, 16, 32, 32, 30, 0, 0, 0, 0, 0, 0, 0, 50, 9, 9, 9, 62, 0, 28, 34, 34, 34, 28, 0, 60, 2, 2, 4, 62, 0, 0, 0, 0, 0, 0, 0, 28, 34, 34, 34, 28, 0, 63, 24, 36, 36, 24, 0, 32, 32, 252, 34, 36, 0, 0, 34, 190, 2, 0, 0, 62, 32, 30, 32, 30, 0, 0, 34, 190, 2, 0, 0, 34, 38, 42, 50, 34, 0, 28, 42, 42, 42, 24, 0, 64, 128, 154, 144, 96, 0, 0, 0, 0, 0, 0, 255, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 248, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 248, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 254, 146, 146, 146, 108, 0, 4, 42, 42, 30, 2, 0, 28, 34, 34, 34, 20, 0, 254, 8, 20, 34, 0, 0, 0, 0])


def get_normalized_bit(value, bit_index):
    return (value >> bit_index) & 1


time_start = time.time()
#first pixels are drawn invisible (some kind of frame in python) so set an offset:
x_offset = 2 
y_offset = 2
x=x_offset
y=y_offset

#display all uC pixels (single screen frame):
byteIndex=0
for j in range(ROWS): #multiple rows
    for i in range(WIDTH): #row
        for n in range(8): #byte
            if get_normalized_bit(diplay_buffer[byteIndex], 7-n):
                img.put("black", (x,y+n))
            else:
                img.put("white", (x,y+n))
        x+=1
        byteIndex+=1
    x=x_offset
    y+=7
time_stop = time.time()
print("Refresh time: ", str(time_stop - time_start), "seconds")    
    
mainloop()
 

Solution

I don’t really use Tkinter, but I have read that using put() to write individual pixels into an image is very slow. So, I adapted your code to put the pixels into a Numpy array instead, then use PIL to convert that to a PhotoImage.

The conversion of your byte buffer into a PhotoImage takes around 1ms on my Mac. It could probably go 10-100x faster if you wrapped the three for loops into a Numba-jitted function but it doesn’t seem worth it as it is probably fast enough.

#!/usr/bin/env python3

import numpy as np
from tkinter import *
from PIL import Image, ImageTk

# INSERT YOUR variable display_buffer here <<<

# Make a Numpy array of uint8, that will become
# ... our PIL Image that will become... 
# ... a PhotoImage
WIDTH, HEIGHT, ROWS = 128, 64, 8
na = np.zeros((HEIGHT,WIDTH), np.uint8)

idx = 0
x = y = 0
for j in range(ROWS):
   for i in range(WIDTH):
      b = display_buffer[idx]
      for n in range(8):
         na[y+n, x] = (1 - ((b >> (7-n)) & 1)) * 255
      idx += 1
      x   += 1
   x  = 0
   y += 7

# Make Numpy array into PIL Image
PILImage = Image.fromarray(na)

border = 10
root = Tk()  
canvas = Canvas(root, width = 2*border + WIDTH, height = 2*border + HEIGHT)  
canvas.pack()  
# Make PIL Image into PhotoImage
img = ImageTk.PhotoImage(PILImage)
canvas.create_image(border, border, anchor=NW, image=img) 
root.mainloop() 

Also, I don’t know how fast your serial line is, but it may take some time to transmit 1024 bytes, so you could consider starting a second thread to repeatedly read 1024 bytes from your serial and stuff them into a Queue for the main process to get() them from.


Also, you could avoid Tkinter altogether, and just use OpenCV imshow() like this:

#!/usr/bin/env python3

import numpy as np
import cv2

# INSERT YOUR display_buffer here <<<

# Make a Numpy array of uint8, that will be displayed
WIDTH, HEIGHT, ROWS = 128, 64, 8
na = np.zeros((HEIGHT,WIDTH), np.uint8)

idx = 0
x = y = 0
for j in range(ROWS):
   for i in range(WIDTH):
      b = display_buffer[idx]
      for n in range(8):
         na[y+n, x] = (1 - ((b >> (7-n)) & 1)) * 255
      idx += 1
      x   += 1
   x  = 0
   y += 7


while True:
  # Display image
  cv2.imshow("Virtual Console", na)

  # Wait for user to press "q" to quit
  if cv2.waitKey(1) & 0xFF == ord('q'):
     break

I decided to have a try with Numba and the time to extract a 128×64 frame dropped to 68 microseconds. Note that the Python has to be compiled first time through, so I did a warm-up run to include the compilation and then measured the second run:

#!/usr/bin/env python3

import numba as nb
import numpy as np
from tkinter import *
from PIL import Image, ImageTk
import time

# Make a Numpy array of uint8, that will become
# ... our PIL Image that will become... 
# ... a PhotoImage
WIDTH, HEIGHT, ROWS = 128, 64, 8
na = np.zeros((HEIGHT,WIDTH), np.uint8)

@nb.njit()
def extract(na,display_buffer):
   idx = 0
   x = y = 0
   for j in range(ROWS):
      for i in range(WIDTH):
         b = display_buffer[idx]
         for n in range(8):
            na[y+n, x] = (1 - ((b >> (7-n)) & 1)) * 255
         idx += 1
         x   += 1
      x  = 0
      y += 7
   return na

# Following is first run which includes compilation time
warmup = extract(na, display_buffer)

# Only time the second run
start = time.time()
na = extract(na, display_buffer)
# Make Numpy array into PIL Image
PILImage = Image.fromarray(na)
elapsed = (time.time()-start)*1000
print(f'Total time: {elapsed} ms')      # Reports 0.068 ms

border = 10
root = Tk()  
canvas = Canvas(root, width = 2*border + WIDTH, height = 2*border + HEIGHT)  
canvas.pack()  
# Make PIL Image into PhotoImage
img = ImageTk.PhotoImage(PILImage)
canvas.create_image(border, border, anchor=NW, image=img) 
root.mainloop() 

Answered By – Mark Setchell

Answer Checked By – Gilberto Lyons (BugsFixing Admin)

Leave a Reply

Your email address will not be published. Required fields are marked *