What’s new in gevent 1.1

Detailed information an what has changed is avaialble in the Changelog. This document summarizes the most important changes since gevent 1.0.3.

Platform Support

gevent 1.1 support Python 2.6, 2.7, 3.3, and 3.4 on the CPython (python.org) interpreter. It also supports PyPy 2.5.0 and above (with best results being obtained on PyPy 2.7.0 and above); PyPy3 is not supported.

Support for Python 2.5 was removed when support for Python 3 was added. Any further releases in the 1.0.x line will maintain support for Python 2.5.

Improved subprocess support

In gevent 1.0, support and monkey patching for the subprocess module was added. Monkey patching was off by default.

In 1.1, monkey patching subprocess is on by default due to improvements in handling child processes and requirements by downstream libraries, notably gunicorn.

• gevent.os.fork(), which is monkey patched by default (and should be used to fork a gevent-aware process that expects to use gevent in the child process) has been improved and cooperates with gevent.os.waitpid() (again monkey patched by default).
• fork-watchers will be called, even in multi-threaded programs.
• The default threadpool and threaded resolver work in child processes.
• File descriptors are no longer leaked if gevent.subprocess.Popen fails to start the child.

In addition, simple use of multiprocessing.Process is now possible in a monkey patched system, at least on POSIX platforms.

Note

All of the above entail forking a child process. Forking a child process that uses gevent, greenlets, and libev can have some unexpected consequences if the child doesn’t immediately exec a new binary. Be sure you understand these consequences before using this functionality, especially late in a program’s lifecycle. For a more robust solution to certain uses of child process, consider gipc.

Monkey patching

Monkey patching is more robust, especially if the standard library threading or logging modules had been imported before applying the patch. In addition, there are now supported ways to determine if something has been monkey patched.

API Additions

Numerous APIs offer slightly expanded functionality in this version. Highlights include:

• A gevent-friendly version of select.poll (on platforms that implement it).
• gevent.fileobject.FileObjectPosix uses the io package on both Python 2 and Python 3, increasing its functionality correctness, and performance. (Previously, the Python 2 implementation used the undocumented socket._fileobject.)
• Locks raise the same error as standard library locks if they are over-released.
• ThreadPool.apply can now be used recursively.
• The various pool objects (gevent.pool.Group, gevent.pool.Pool, gevent.threadpool.ThreadPool) support the same improved APIs: imap and imap_unordered accept multiple iterables, apply raises any exception raised by the target callable.
• Killing a greenlet (with gevent.kill() or Greenlet.kill) before it is actually started and switched to now prevents the greenlet from ever running, instead of raising an exception when it is later switched to.
• Almost anywhere that gevent raises an exception from one greenlet to another (e.g., Greenlet.get), the original traceback is preserved and raised.

Compatibility

This release is intended to be compatible with 1.0.x with minimal or no changes to client source code. However, there are a few changes to be aware of that might affect some applications. Most of these changes are due to the increased platform support of Python 3 and PyPy and reduce the cases of undocumented or non-standard behaviour.

• gevent.baseserver.BaseServer deterministically closes its sockets.

  As soon as a request completes (the request handler returns), the BaseServer and its subclasses including gevent.server.StreamServer and gevent.pywsgi.WSGIServer close the client socket.

  In gevent 1.0, the client socket was left to the mercies of the garbage collector. In the typical case, the socket would still be closed as soon as the request handler returned due to CPython’s reference-counting garbage collector. But this meant that a reference cycle could leave a socket dangling open for an indeterminate amount of time, and a reference leak would result in it never being closed. It also meant that Python 3 would produce ResourceWarnings, and PyPy (which, unlike CPython, does not use a reference-counted GC) would only close (and flush) the socket at an arbitrary time in the future.

  If your application relied on the socket not being closed when the request handler returned (e.g., you spawned a greenlet that continued to use the socket) you will need to keep the request handler from returning (e.g., join the greenlet) or subclass the server to prevent it from closing the socket; the former approach is strongly preferred.

• gevent.pywsgi.WSGIServer ensures that headers set by the application can be encoded in the ISO-8859-1 charset.

  Under gevent 1.0, non-bytes headers (that is, unicode since gevent 1.0 only ran on Python 2) were encoded according to the current default Python encoding. In some cases, this could allow non-Latin-1 characters to be sent in the headers, but this violated the HTTP specification, and their interpretation by the recipient is unknown. Now, a UnicodeError will be raised.

  Most applications that adhered to the WSGI PEP, PEP 3333, will not need to make any changes. See issue #614 for more discussion.

• Under Python 2, the previously undocumented timeout parameter to Popen.wait (a gevent extension ) now throws an exception, just like the documented parameter to the same stdlib method in Python 3.

Next page: API reference