This is something I sent out to the gimp list in December, annotated
with some responses I got. My ideas have evolved a bit since
then. I a better idea of what the dialog control dialog should
look like, and I think the user should be able to switch between 
three modes for each device:

1. Core pointer only
2. Mapped to a window
3. Core pointer but switching to program control
   when the pointer enters a window

[ update: 4/1/97. Mode 1 is now
   1. Controlled by gxid (autoswitched core pointer) ]

Switching the core pointer when necessary should probably be
triggered by getting events for that device.

Hopefully I'll implement some of this for xink soon.

                      Owen Taylor <owt1@cornell.edu>

LE = Larry Ewing <lewing@isc.tamu.edu>
PS = Paul Sargent  <PaulS@3dlabs.com>

Proposal for an XInput extension interface to the GIMP
------------------------------------------------------

Abstract:

A system for handling multiple devices with capabilities beyond
that of the core X pointer is introduced.  A small number of
additional functions will be implemented in GDK to provide a
low level interface.  The concept of drawing tool location is
separated from the concept of the X pointer location, which
will involve some changes to the existing drawing tools.  A
new dialog provides a user interface for switching between
devices and controlling their properties.  Each device can have
its own current tool, brush, and paint color.

Background:

New advances on several fronts have made available to the
average user input technologies more conducive to artistic
expression than the standard mouse.  In particular, these
include the introduction by Wacom (http://www.wacom.com), of an
inexpensive (< $150) pressure sensitive drawing tablet, the
Artpad II, and the inclusion of XInput extension support for
devices including drawing tablets in XFree86 v3.2.  Some limited
use of these devices can be made using existing versions of the
GIMP by having the device control the X pointer.  However, this
does not allow the use of features such as pressure
sensitivity, high resolution, or multiple input types (such as
an "eraser" on a tablet stylus).  Applications need
modifications to support these features.

User Interface:

The major change to the basic abstractions of the GIMP needed
is that the X pointer and the position of the drawing tool
should to be treated as as independent concepts.  This seems to
be necessary from a practical point of view, because it isn't
possible to get detailed information about the XInput device if
it is the core pointer, and we can't move the core pointer
ourselves.  (using XWarpPointer) quickly enough to track the
XInput device.

PS> What Larry suggested (I think) was that we switched the tablet between
PS> Xinput mode and Xcore mode upon entering and exiting the drawing window.
PS> Then X does the moving of the Core pointer, not Gimp. Then the user
PS> doesn't have to put down the stylus and pick up the mouse when wanting
PS> to select a different tool.

The above is also desirable independent of the practicality -
First, it one has a canvas occupying only part of the screen,
one doesn't have to waste tablet real estate for the rest of
the screen (Throughout this document, it is assumed that the
input device is a drawing tablet - joysticks seem to have
limited potential for drawing). 

PS> Agreed, and that was my major gripe against the system I describe, so we
PS> decided to have it user switchable. Lock to Canvas or to whole screen,
PS> at the touch of a button or key.

Second, we can use resolutioninformation better than one pixel.

This change should be invisible to the user in the case when
the only input device is the core pointer - although there is a
need for a graphical representation of the drawing cursor
location if it is different from the core pointer, this can be
omitted when they are identical.  A good possiblity for this
graphical representation is to lightly overlay or XOR an image
of the current brush onto the drawing (though this would work
poorly for single pixels brushes).

LE> Someone correct me if I'm wrong but I vaguely remember that there is a
LE> software patent on XORing in a pointer?

OT> Aargh.  Software patents.  But this doesn't actually have to 
OT> be an XOR since reversibility doesn't matter.  I can't imagine it's
OT> possible to patent every sort of transparent overlay.

The control of the drawing cursor would work as follows:

I. If there is an active device other than the core pointer,
then the drawing cursor is controlled by this device.  Which
image the cursor appears in depends on the window receiving the
events from the device - this is determined in a manner very
similar to the keyboard focus.

II. If no other device is active, then the drawing cursor
follows the pointer if it is a area where this make sense,
otherwise the cursor retains its last position (or possibly
vanishes).

LE> In this case it probably shouldn't even be shown at all.  If you want to
LE> show the brush type etc. then change the core pointer's pixmap.

A new dialog box would be added to allow the user to control
which device is active, properties of the device (sensitivity,
button mappings, etc.), and which device controls the X
pointer.  It is probably also desirable to set up convenient
key or button mappings to allow rapid switching of the device
controlling the X pointer.  Instead of reaching for the mouse
when they want to access the menus, users can simply toggle
their drawing tablet to control the core pointer, perform the
operation, then toggle it back.

LE> Hot keys to switch the mapping will be important because otherwise you'll
LE> constantly be swaping the mouse and tablet in your hand.  A few of us
LE> thought this through a little and there are a lot of difficulties involved 
LE> in trying to do it much more cleanly.  An additional useful possibility is
LE> having a popup menu that allows you to toggle extension/core setting on
LE> that device, and that allows you to set the current tool etc.    

OT> Good idea. Perhaps whatever triggers the popup menu should
OT> temporarily switch the current device back to the core pointer.
OT> I'd like to avoid having to deal with the interaction of non-core
OT> devices with GUI elements, if possible.

The mapping of the devices parameter space to the image needs
to be fairly flexible - first, when the user zooms in on a
window, the device should be mapped only to the portion of the
image that is visible.  Second, when the entire image is
visible, it should be possible for the user to go a bit beyond
the edge of the image so that they can, for instance, select
right up to the border.

PS> You also need to think about the aspect ratio of the device, Vs the
PS> aspect ratio of the canvas. If your canvas is square and the tablet is
PS> 4x3 you want to use a square region in the centre of the tablet and not
PS> the whole thing.

Each device should retain its own current tool, color, brush,
and other relevant drawing parameters.  This allows a great
deal of convenience to be obtained from devices like the
"erasing stylus" mentioned above.

PS> Do you mean ala Fractal Design Painter? If so Agreed. For those who
PS> don't know this allows any tool to be put on any device, so with tablets
PS> that have special erasers, you could have anything you like on it and
PS> not like photoshop where it must be an eraser.

Little change needs to be made to the drawing tools to take
advantage of the additional capacities of an input device.
There is already support for sub-pixel positioning for
drawing tools, and the pressure coordinate of input can
be mapped to tool pressure for devices such as the airbrush,
convolve tool, and paintbrush, where this concept is already
implemented, or to the brush opacity where it is not.

On the other hand, once the basic support for XInput devices is
implemented, there is the potential for adding even more
sophisticated drawing tools such as found in commercial
programs like Fractal Design Painter.  It may be also desirable
to add a plug-in-like capability for user-created tools,
(probably using shared libraries for speed reasons).  See
also [Ewing].

LE> This is really important in the long term.  It would allow us to
LE> implement large ximput support things cleanly without bloating the core
LE> of the gimp.  I think S&P ( :) ) have this in mind for the future.

Programming Details:

Some small additions need to be made to the GDK library which
provides a thin wrapper around the X libraries for the GIMP.
The main changes that need to be made are the addition of
wrapper functions for the XInput functions for listing,
opening, and selecting devices.  Some changes will also need to
be made to the routines for getting X events to handle
correctly the new event types generated by extension devices.
Almost no changes should be needed to the GTK widget library,
since the interpretation of XInput device information is very
application dependent.

The changes that have to be made to the existing structure of
the GIMP itself are pretty small.  When events for a gdisplay are
handled, the translation from core pointer position and XInput
device events to drawing cursor location in image coordinates
needs to be done.  The drawing tools themselves then never see
screen coordinates.  (This should not be a problem, since they
all seem to call gdisplay_untransform_coordinates as their
first action).  Presumably, instead of being passed an actual
motion event structure, they would be passed a structure with
similar information, but in image coordinates.  

Other than this, the graphical display of the drawing cursor
location also has to be implemented, at least rudimentary use
of pressure information needs to be implemented, along with the
above mentioned dialog for selecting device function.

Possible future enhancements:

1. Fancier drawing tools, with more sophisticated use of
pressure (and tilt, for those so lucky)
PS> On Wacom tablets, all but the most basic tablet has tilt. I'd like it to
PS> be one of the first thing we think about, not one of the last.
2. User definable drawing tools

3. More support for input devices other than drawing tablets
(but major conceptual difficulties should be addressed now).

LE> I'd also really like to do some motion history support.  The patches to
LE> prevent event compression are nice, but if the device can report the full
LE> path then we should use it.  
 
Conclusion:

The implementation of XInput device support for the GIMP seems
to be feasible with only a moderate amount of work, and no
changes to the core algorithms.  If done properly, such support
would help make the GIMP as much of a heavyweight in the realm
of freehand artistic expression as it is in the realm of image
manipulation.

References:

[Ewing] Larry Ewing. Drawing With the GIMP.
(http://www.isc.tamu.edu/~lewing/gimp/)

Another person interested in adding XInput capabilities to the
GIMP. With some drawings showing what you can do with a
graphics tablet, even without specific support.
