The question of how to speed up importing of Python modules has been asked previously (Speeding up the python "import" loader and Python — Speed Up Imports?) but without specific examples and has not yielded accepted solutions. I will therefore take up the issue again here, but this time with a specific example.
I have a Python script that loads a 3-D image stack from disk, smooths it, and displays it as a movie. I call this script from the system command prompt when I want to quickly view my data. I’m OK with the 700 ms it takes to smooth the data as this is comparable to MATLAB. However, it takes an additional 650 ms to import the modules. So from the user’s perspective the Python code runs at half the speed.
This is the series of modules I’m importing:
import numpy as np import matplotlib.pyplot as plt import matplotlib.animation as animation import scipy.ndimage import scipy.signal import sys import os
Of course, not all modules are equally slow to import. The chief culprits are:
matplotlib.pyplot [300ms] numpy [110ms] scipy.signal [200ms]
I have experimented with using
from, but this isn’t any faster. Since Matplotlib is the main culprit and it’s got a reputation for slow screen updates, I looked for alternatives. One is PyQtGraph, but that takes 550 ms to import.
I am aware of one obvious solution, which is to call my function from an interactive Python session rather than the system command prompt. This is fine but it’s too MATLAB-like, I’d prefer the elegance of having my function available from the system prompt.
I’m new to Python and I’m not sure how to proceed at this point. Since I’m new, I’d appreciate links on how to implement proposed solutions. Ideally, I’m looking for a simple solution (aren’t we all!) because the code needs to be portable between multiple Mac and Linux machines.
you could build a simple server/client, the server running continuously making and updating the plot, and the client just communicating the next file to process.
I wrote a simple server/client example based on the basic example from the
socket module docs: http://docs.python.org/2/library/socket.html#example
here is server.py:
# expensive imports import numpy as np import matplotlib.pyplot as plt import matplotlib.animation as animation import scipy.ndimage import scipy.signal import sys import os # Echo server program import socket HOST = '' # Symbolic name meaning all available interfaces PORT = 50007 # Arbitrary non-privileged port s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) s.bind((HOST, PORT)) s.listen(1) while 1: conn, addr = s.accept() print 'Connected by', addr data = conn.recv(1024) if not data: break conn.sendall("PLOTTING:" + data) # update plot conn.close()
# Echo client program import socket import sys HOST = '' # The remote host PORT = 50007 # The same port as used by the server s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) s.connect((HOST, PORT)) s.sendall(sys.argv) data = s.recv(1024) s.close() print 'Received', repr(data)
you just run the server:
which does the imports, then the client just sends via the socket the filename of the new file to plot:
python client.py mytextfile.txt
then the server updates the plot.
On my machine running your imports take 0.6 seconds, while running
client.py 0.03 seconds.
Answered By – Andrea Zonca
Answer Checked By – David Goodson (BugsFixing Volunteer)