Saturday, August 20, 2011

Renderscript: Components of a 3d scene

For anyone looking into renderscript who has never done any 3d graphics before, this is for you!  There is a lot of new terminology, and a whole lot of math involved.  Scared yet?  Don't be, the set of components involved in rendering a 3d scene is finite and well defined.

Rendering Pipeline
Rendering a scene simple means that you are transforming a 3D field of objects into a 2D image.  I found this easy to understand drawing that should help you to visualize the rendering pipeline.  Credit to these Waterloo university course notes here.

This is what it's all about.  Objects in a scene, comprised of primitives.  There is some key terminology used when we discuss how to build an object in 3d space.

  • Plane This is a a one dimensional slice in space that runs along one of three axis, x, y, or z.
  • Coordinate This is an element that is used to describe a precise point in a single plane.
  • Vertex This is a set of coordinates that are used to describe a precise point in a 3d space.  A vertex is described in the x, y, and z planes.  Typically x represents the horizontal plane, y represents the vertical plane and z represents the depth plane.
  • Primitive A primitive is an atomic geometric object.  ie: points, planes, lines, circles, triangles, spline curves.  For more on primitives see this wiki page.
An object is composed of a set of vertices, and a type of geometric primitive that describes how the vertices should be connected to make a shape.

A models coordinate system (MCS) needs to be transformed into your worlds coordinate system (WCS) using a model transformation matrix.

A 3D world scene will contain your entire scene.  However, the user may not need to see your entire scene on their device.  In order to see smaller sections of your world scene, you need to create a view scene by transforming your world scene to your view scene using a view transformation matrix.  This is a transformation from your WCS to your view coordinate system (VCS).

In order to display your 3D view of your scene on your 2D screen, first we need to calculate what parts of your 3D view will actually be seen.  This is done through a planer projection.  There are two types of 3D planer projections, the one we would normally be using in a renderscript context will be perspective projection.  Basically perspective projection is the idea that objects that are further away look smaller than objects that are closer to the viewer.

The components we need to define in renderscript in order to project a 3D scene onto a 2D screen are as follows:
  • Camera The camera is positioned somewhere in the 3D scene and oriented towards a particular part of the scene.  We also need to define the cameras field of view, which determines how much of the scene is seen by the camera.  In order to actually see something on your 2D screen, you will finally need to define a plane with which we project our scene through.  You can see some drawings that depict this description here.
  • Model This is your 3D model that you want projected.
  • Plane through which your camera projects your scene.
Once your scene is projected into a normalized device coordinate system (NDCS) it is ready for rasterization into the final 2D image in in your screens coordinate system (SCS).  The process of rasterization involves transforming your 2D scene into it's pixel representation on your screen.

This is just a high level view of the basics of 3D rendering, but you need to understand this before digging deeper into the details of how all of this works.  It involves a lot of complicated vector and matrix math.  For more details, have a look at these wikipedia articles.

Thursday, August 18, 2011

Renderscript: A First Look

What is Renderscript?
Renderscript is a C99 based language used for 3D rendering and compute API's at the native level on an Android device.  It's a new API provided in the Honeycomb SDK level 11.

When should I use Renderscript?
When you need a high performance graphics rendering solution in your application and/or a high performance compute language for doing resource intensive operations.  Renderscript doesn't require you to write any JNI code to interface with your renderscript code.  The interface between your renderscript and your SDK code is generated for you.  The trade offs are,  you have a reduced feature set compared to using OpenGL with the NDK and you can't allocate memory in your renderscript, it must be allocated from your SDK code.

How is it different than OpenGL?
OpenGL can be used in two different ways in Android.
  • OpenGL API's can be used to produce 3d graphics using the Android SDK.  This is a good choice when performance isn't critical and you don't need the added complexity of using the NDK.
  • OpenGL with the NDK offers you the most flexability and features of OpenGL and is a good choice for graphics intense applications.  However, you have the added complexity of the NDK and you are required to write your own JNI to use your NDK code.
Renderscript components
An app that uses renderscript is composed of three required components.

Renderscript component
This is your actual Renderscript code contained in a .rs file. Renderscript code is C99 based but only has a small set of functionality so as to ensure portability across different CPUs.  You can also create renderscript header files (.rsh) where you can define your structures and function stubs.  You can have multiple .rs and .rsh files.  You can find the provided renderscript header files in the <sdk_root>/platforms/android-11/renderscript directory of the Android SDK.

Reflected component
This component is set of classes generated for you by the Android build tools and are used to interface with your renderscript component.  This includes hooks to the functions and variables in your renderscript code.  Also provided are methods for allocating memory to the pointers defined in your renderscript.
For each .rs file you've created, a class will be generated with the name ScriptC_renderscript_filename. This is the java representation of your .rs that you can call from your SDK code. Code that will be reflected in this generated class is as follows:
  • Non-static functions
  • Non-static global variables are accessed with generated set_ and get_ methods, if a variable is marked as const in your renderscript, then only a get_ method is generated.  It should be noted that the last value set to a variable through the reflected set_ method, is the value that will be returned through the get_ method, even if that variable is changed inside your renderscript during execution.
  • Global pointers are accessed in the reflected component by special methods generated called bind_pointer_name. This will allocate memory for the pointer both in the SDK code and in your renderscript.
Android Framework component
This is the usual Android framework where you have access to the android.renderscript classes as well as the usual things like Activity lifecycle, touch events, etc.  You will access your renderscript from here through the reflected components.