236 Project

Painty Paint!

proposal:

There are several areas in which this paint model may be extended. 

• The paint volume holding capacity is limited, and causes unnatural results when attempting to apply paint to an already saturated canvas.
• The paint is rendered currently with only a simple embossing over a color texture which has been alpha blended
• More attributes can be added to the paint model including 
• variable amounts of supporting media (oil)
• bristle indentation direction (anisotropic effects from bristles)
• The heightfeild of the paint is not used when depositing paint
• The paint transfer rate is not physically or time based.
• Paint does not travel through the brush tip as it should. 
• ...

progress update 1

Overview

The Bug, and how it is impeding progress

The current error is that the texture returning from the canvas is somehow ‘scrambled’ slightly. The effects look, to the human eye, very much like the zippering effects of Z-Fighting. Z-Fighting, however, is not the cause of the problem, as there are no coplanar polygons. 

The mixing of the texture is acceptable as it is currently used with the surface layer color & volume. The effects are not very noticeable because these attributes change very rapidly. I.E. the surface paint has a very high turnover rate, and in the process does not have great variation in color, it is therefore not very noticeable. The volume errors are not very noticeable due to the large volume of ‘reservoir’ paint just behind the surface layer. The effects on the reservoir volume are slightly noticeable as the brush depletes it’s supply of paint, the trailing paint at the end appears slightly ‘chunky’. Most users would not notice this. 

However, the bug is serious if one were to update a more sensitive attribute layer, such as the reservoir paint color texture. This texture is expected to change only slightly over a long period of time. The scrambling bug quickly blends the reservoir colors. This has the effect of the paint mixing thoroughly on the brush, destroying a ‘complex blend’ which may have been picked up. 

Many of the proposed additions to the paint model require additional attribute channels. For example, many require carrying more than 8 bits of information for the volume of paint stored on the brush. (the 8 bits comes from the 8 bits per channel supported by hardware rasterization.)  A second attribute channel would provide 16 bits, by adding 8 higher significance bits. These bits, due to their high significance, would be adversely affected by the scrambling bug. In a similar way, other required attribute channels will simply not function in the presence of the scrambling bug. 

More time must be spent to correct this bug, or find the flaw in the approach used. Or, alternative methods may be used for the paint transfer.

Alternative paint transfer method number 1: software implementation

Alternative paint transfer method number 2: texture tagging and meshing

This approach is slightly more expensive than the current implementation (involving a lookup for each texture when mapped down, and a re-meshing for the transfer back up). The implementation of this method should not require as much work as the alternative method number 1.

Explicit additions to the paint system under consideration

Smooth path interpolation
As the user moves the input device rapidly, their movements are discreteized. The current paint system linearly interpolates between these positions. A better solution may use a curve interpolation method to provide greater continuity. 
Implementation complexity low

Anisotropic effects
The direction for which a brush stroke has been laid may be recorded at each texel on the canvas. When rendering, this information can be used with an anisotripic light model to simulate the fine groves left by the bristles of the brush. This effect is clearly noticeable in some real paintings.
May not be very obvious until light and eye positions are easily movable (currently fixed for many optimization reasons)
Implementation complexity low

Better bump mapping
Currently an embossing technique with fixed light position is used. A more versatile bump mapping technique, such as Blinn bump mapping, would provide a more convincing rendering. This could be implemented either in software or hardware. Hardware implementations are rumored to be very difficult.
Implementation complexity medium

Better specular light model
Current light model is only the embossing. A better specular model may allow for the indication of ‘wet’ paint by higher specular term. However, this will likely slow rendering quite a bit, and may be passed over in favor of a much more complex rendering method, such as described below.
Implementation complexity low

Kubelka Munk color model
The paint industry researched the precise optical properties of oil paints, and the Kubelka Munk model provides a method to render thick layers of paint accurately. Subsurface scattering theory should be able to be combined with this method to provide for rendering of thin layers of oil paints on top of each other. This would ideally provide a more realistic look to the paint model. It would require a change to the ‘pigment’ model of the paint model, however the rendering complexities far outweigh the pigment model changes.
Implementation complexity high

Multilayer paint model
Currently a dry and wet layer are simulated. Allowing for additional wet layers, possibly arbitrarily more, would allow for a more accurate model. Complex actions can be performed with this additional state information, such as layering several colors of paint and then gouging into them. Adding just one more layer, to reach a model with one dry layer and two wet layers, causes problems with partial drying. It is not readily apparent how these layers would be partially dried, while maintaining their distinct layers. With respect to an unlimited number of layers to deposit onto, it is clear that a truly unlimited number can not be provided. However, a few layers could be provided, and if the user applies more paint then the previous layers could be pushed down, in a FIFO manner. The oldest layer would become ‘burried’, or lost, possibly merged into the dry layer. If this were done locally per texel, a counter could be incremented to denote the layer shifting. On the other hand, if the entire canvas was shifted down the user would experience a noticeable pause. Depositing the paint in this model is only slightly expensive, but blending with it may be much more complicated. Gouging would require moving large volumes of the paint, and this model does not support that easily. It is not clear that a distinctly layered approach is the proper solution for the desired attributes.
Implementation complexity high

Movable paint volumes
Currently paint moves between the brush and canvas only. It does not move across the canvas, unless picked up and redeposited by the brush. True paint can be pushed in large volumes across the canvas by the brush, where the pain piles up in a ‘front’. To implement this a data structure must be constructed which allows paint volume to move in three dimensions, and a simulation algorithm must solve for the volumes of paint moved. A full navier-stokes simulation would be expensive, and is likely overkill for this application. A reduced simulation may be able to use the constraints of relatively thing volumes moving very viscously upon a flat surface. Pressure of the brush can be determined from the penetration of the brush, and this used to determine how much paint is displaced by a cellular pressure and viscosity based transport. 
Implementation complexity high

Bristle gouging into paint volume
A texture on the brush may be used to indicate texels that are under higher pressure than others, allowing for individual bristles to gouge slightly deeper into the paint. Implementing this across the entire brush surface is overkill, as this effect will only be seen by the last portion of the brush to be in contact with a part of the canvas. This will have different implementations for the current, distinct multiplayer, and voxel canvas models. The effects simply can not be seen at the current resolution of the canvas – and higher resolutions run too slowly. 
Implementation complexity high

Paint transfer based on canvas height field
Peaks on the canvas should catch paint before valleys, and under high pressure the peaks should be scraped clean. A huristical approach may use image processing techniques on the height field of the canvas to determine rate of transfer for each texel. 
Implementation complexity medium

Multiresolution canvas
Upon detail work, a portion of the canvas may increase it’s resolution, and be inset into the lower resolution of the main canvas. This allows for dramatic scale differences in paint strokes, and is just really hard and therefore attractive. Ideally, the higher resolution portions could still be painted over with low resolution techniques when they are not being focused on, allowing for greater simulation speeds. User interface for this may be very difficult – as they would need to be able to not only specify when to increase the resolution of an area, but also clearly see which areas of the canvas are at specific resolutions. A method for detecting when a higher resolution is needed is not apparent at this moment. 
Implementation complexity high

Plans for implementation

After that time, the outcome of the Siggraph paper will possibly affect what is attempted. Most likely, however, is the voxel style canvas volume representation with very simplified paint moving simulation. 
 
 

progress update 2

Overview

Various papers on multiresolution image techniques were published, mostly in the mid ninties. The implementation choosen for this project most closely resembles the ‘sparse quadtree’ presented by
Berman, Bartell, and Salesin
Multiresolution Painting and Compositing (1994)
 

Specific Work accomplished since last update:

• the DAB system did not properly store the volume of paint – the implementation was faulty, and a correct implementation of a relative height field was put into the system.
• The topic of multiresolution image was choosen from the many options available.
• Various papers on the topic were read, and the many approaches considered.

Specifics of the implementation:

• Multiresolution image format, containing a flexible pixel format (allowing for the many attributes required for the painting software)
• A sparse quadtree approach. Each node in the quad tree has the same number of texels, however child nodes are at ½ the scale of their parents. 
• At a specific zoom level, a screen sized cache will be allocated to allow the user to work rapidly. 
• A lazy approach will be taken for propagating that changes the user makes up and down the quadtree. Only at zoom level change, or panning about the image, will the changes be propagated. 
• The user will be given nagivation via the haptic input device. Pressing the button on the stylus will enable movement, which will correspond to the 3DOF position movements. 
• As a side note, this will require a complete replacement of the image format datastructures, which are currently implemented in a C style, with distributed responsibility.