Java 3D panoramas

This is a panorama viewer: six photos, one for each direction, are put on the inside of a cube, and the viewpoint is placed at the centre of it, so you can look all the way round and straight up and down. It's a Java applet with no 3D library underneath, drawing every pixel itself into an array of ints.

The hard part is drawing the photos without them bending. Each one covers a whole wall that runs from in front of you round to the side of you, so it needs perspective-correct texture mapping, and it has to cope with walls that are partly behind you without any clipping.

There are three applets, each one a step on from the last: a spinning cube of flat-coloured triangles, the panorama viewer, and the same panorama with the cube unfolded into six views at once.

Test Java cube rendering

This is a barebones renderer for one object, with a software solid triangle rasteriser. A cube of twelve triangles turns about the vertical, tipped towards you, with every triangle filled in its own colour.

Panorama viewer

Placing the viewpoint inside the cube and making the faces of the cube textured. The six photos are 256x256 JPEGs, one for each face:

Unfolded panorama

Instead of rotating the cube, the texture can be warped and the cube and viewpoint can remain static. The panorama is drawn six times, once looking out through each face of the cube, and the six views are laid out as an unfolded cube. Dragging turns the panorama inside it, so the pictures slide from one face across to the next.

This version runs at the top of the page.

The screen

The screen buffer is provided by TinyPTC, a tiny library by Gaffer that turns an int[] into an image in a Java applet, the same as in Java 3D pillars. It is an int[] linear RGB pixel array, one int per pixel and one row after another, so the pixel at (x, y) is buf[y*w + x].

The applet class, House, extends tinyptc. When the applet starts, TinyPTC starts a thread that reads the applet's size and calls main(width, height), and that never returns: it makes the buffer and loops forever, drawing a frame and handing it over.

public void main(int width, int height)
{
  int size = width * height;
  int buf[] = new int[size];
  int seed = 0x12345;

  Z3D_Scene scene = new Z3D_Scene(buf, width, height);

  while (true)
  {
    scene.render();
    update(buf);
  }
}

There's no sleep in the loop, so it runs as fast as the machine will go, and the cube spins faster on a faster machine. update passes the whole array to the image as one finished frame and paints the image onto the applet:

public synchronized void update(Object pixels)
{
    // check consumer
    if (_consumer!=null)
    {
        // copy integer pixel data to image consumer
        _consumer.setPixels(0,0,_width,_height,_model,(int[])pixels,0,_width);

        // notify image consumer that the frame is done
        _consumer.imageComplete(ImageConsumer.SINGLEFRAMEDONE);
    }

    // paint
    paint();
}

The image's colour model takes red, green and blue from the bottom 24 bits of each int and has no alpha, so whatever is in the top 8 bits doesn't matter. That comes in handy later.

Matrices and vectors

These come from idx3d, the Java 3D engine by Peter Walser: three of its classes, idx3d_Matrix, idx3d_Vector and idx3d_Math. Only a few parts are needed. The cube is turned with a matrix, built with rotateSelf and moved away from the eye with shift, and the points are run through it by hand, using the matrix's fields:

rx = -0.5f;
ry += 0.01f;

//  Transform

float x, y, z;
idx3d_Matrix m;

m = new idx3d_Matrix();
m.rotateSelf(rx, 0, 0);
m.rotateSelf(0, ry, 0);
m.shift(0, 0, 4);

for(i=0; i<numPoints; i++)
{
  x = point[i][0];
  y = point[i][1];
  z = point[i][2];

  tpoint[i][0] = x*m.m00 + y*m.m01 + z*m.m02+ m.m03;
  tpoint[i][1] = x*m.m10 + y*m.m11 + z*m.m12+ m.m13;
  tpoint[i][2] = x*m.m20 + y*m.m21 + z*m.m22+ m.m23;
}

The order matters. rotateSelf multiplies the new rotation in on the right, so it's applied to the points first, and rotate and shift multiply on the left, so they're applied last. Here the spin about y happens first, then the tip about x, then the move away, so the cube spins about its own axis and stays tipped the same way towards you. The unfolded panorama uses rotate to turn the camera after everything else.

idx3d's sin and cos look the angle up in a table of 4096 entries rather than calling Math.sin, so angles go in steps of about 0.0015 radians. idx3d_Math.clearBuffer clears the screen by setting the first pixel, then copying it into the second, those two into the next two, and so on, doubling each time with System.arraycopy. idx3d_Vector is only there because idx3d_Matrix refers to it.

Flat triangles

The cube is eight points at ±1 and twelve triangles, two to each face. Once the points are turned, they're projected onto the screen by dividing by z:

spoint[i][0] = w*0.5f + w*0.5f * x / z;
spoint[i][1] = h*0.5f + w*0.5f * y / z;
spoint[i][2] = z;

The width is used for both, so the pixels come out square. With the cube's centre 4 units away, a face turned square on to you is 3 units from the eye, and its corners, 1 unit to the side, land w/6 from the centre: 92 pixels in the 550-pixel applet here. y isn't flipped, so +y is down the screen.

Each triangle gets a colour made from its number, tri_col(i*0x87638723), which gives twelve unrelated colours: 638723, C70E46, 2A9569 and so on. That's why every face is two-tone. Triangle 0 comes out as 0, black, but it's on the face that the tip keeps turned away from you, so it's never seen.

tri_col fills one triangle a line at a time:

public void tri_col(int col)
{

  float t;
  int x, y, y1, y2, x1, x2, o, ti;
  float c1, c2;
  float dc;

  //  Don't draw if backfacing

  if((tri_x2-tri_x1)*(tri_y3-tri_y1) > (tri_x3-tri_x1)*(tri_y2-tri_y1)) return;

  // Sort points

  if(tri_y1 > tri_y2) { t=tri_y2; tri_y2=tri_y1; tri_y1=t;  t=tri_x2; tri_x2=tri_x1; tri_x1=t; }
  if(tri_y2 > tri_y3) { t=tri_y3; tri_y3=tri_y2; tri_y2=t;  t=tri_x3; tri_x3=tri_x2; tri_x2=t; }
  if(tri_y1 > tri_y2) { t=tri_y2; tri_y2=tri_y1; tri_y1=t;  t=tri_x2; tri_x2=tri_x1; tri_x1=t; }

  // Clip top point to screen and pixel

  y1 = (int)tri_y1; if(y1>=h) return; if(y1<0) y1 = 0;
  y2 = (int)tri_y3; if(y2<0) return; if(y2>h) y2 = h;

  c1=0; c2=0;

  for(y=y1-1; y<y2+1; y++)
  {
    if(y >= tri_y1 && y < tri_y3)
    {
      if(y >= tri_y1 && y < tri_y2)
      {
        x1 = (int)(tri_x1 + (tri_x3-tri_x1) * (y-tri_y1) / (tri_y3-tri_y1));
        x2 = (int)(tri_x1 + (tri_x2-tri_x1) * (y-tri_y1) / (tri_y2-tri_y1));
      }
      else
      {
        x1 = (int)(tri_x1 + (tri_x3-tri_x1) * (y-tri_y1) / (tri_y3-tri_y1));
        x2 = (int)(tri_x2 + (tri_x3-tri_x2) * (y-tri_y2) / (tri_y3-tri_y2));
      }
      if(x1 > x2)
      {
        ti = x2; x2 = x1; x1 = ti;
        t = c2; c2 = c1; c1 = t;
      }
      if(x1 < 0) x1 = 0;
      if(x2 > w) x2 = w;

      if(x2 > x1) {
        dc = (c2-c1)/(x2-x1);

        o = y*w + x1;

        for(x = x1; x < x2; x ++) buf[o++] = col;
      }

    }
  }

}

The first test is the back-face check. It's the sign of the cross product of two edges on the screen, which says whether the corners go round clockwise or anticlockwise, and a face turned away from you has them the other way round. On a convex shape like a cube, throwing those away is all the hidden surface removal it needs: the faces left can never overlap, so there's no sorting and no depth buffer.

Then the corners are sorted from top to bottom. Every line of the triangle crosses the long edge from the top corner to the bottom one, and one of the two short edges, 1 to 2 above the middle corner and 2 to 3 below it. Each end is worked out from scratch on every line, with a divide, and the line between them is filled. c1, c2 and dc are swapped and worked out but never used: room for shading across the line that was never filled in.

Inside the cube

For the panorama, each point carries five numbers, x, y, z and the texture coordinates u and v, and the faces are squares of four corners rather than triangles:

face[0][0] = 3; face[0][1] = 0; face[0][2] = 1; face[0][3] = 2; // 0
face[1][0] = 2; face[1][1] = 1; face[1][2] = 5; face[1][3] = 6; // 1
face[2][0] = 3; face[2][1] = 2; face[2][2] = 6; face[2][3] = 7; // 2
face[3][0] = 6; face[3][1] = 5; face[3][2] = 4; face[3][3] = 7; // 3
face[4][0] = 0; face[4][1] = 3; face[4][2] = 7; face[4][3] = 4; // 4
face[5][0] = 1; face[5][1] = 0; face[5][2] = 4; face[5][3] = 5; // 5

...

tex[0] = getPixels(applet, "top.jpg", 256, 256);
tex[1] = getPixels(applet, "front.jpg", 256, 256);
tex[2] = getPixels(applet, "right.jpg", 256, 256);
tex[3] = getPixels(applet, "bottom.jpg", 256, 256);
tex[4] = getPixels(applet, "back.jpg", 256, 256);
tex[5] = getPixels(applet, "left.jpg", 256, 256);

The view starts out looking along +z, so the first thing you see is back.jpg.

The photos are loaded with the applet's getImage, from the same folder as the page, and PixelGrabber copies each one into an int[] of 65,536 pixels, in the same 0xRRGGBB form as the screen. So drawing a texel is just copying an int. The 210x18 logo images are loaded the same way.

The matrix only turns now: the shift is commented out, so the eye stays at the centre of the cube, and the y in the projection is flipped so that up is up. Every face is split into 4x4 smaller squares, which are cut up in 3D, before projecting, by interpolating all five numbers between the corners:

//- Texture co-ords

p1[3] =   0; p1[4] =   0;
p2[3] = 256; p2[4] =   0;
p3[3] = 256; p3[4] = 256;
p4[3] =   0; p4[4] = 256;

//- Subdivide into 4x4

int i, j;

for(j=0; j<4; j++)
{
  for(i=0; i<4; i++)
  {
    //  Calculate square

    interpolate(p1, p2, (i)*0.25f, tp1);
    interpolate(p4, p3, (i)*0.25f, tp2);
    interpolate(tp1, tp2, (j)*0.25f, ip1);

    ...

    //  Screen co-ords

    float scale = 0.4f;//0.45f;


    ip1[0] = w*0.5f + w*scale * ip1[0] / ip1[2];
    ip1[1] = h*0.5f - w*scale * ip1[1] / ip1[2];

    ...

    //  render square

    renderQuad(index, ip1, ip2, ip3, ip4);
  }
}

With a scale of 0.4 of the width, the view is about 103 degrees across, and 85 degrees top to bottom in the 550x400 applet here.

The reason for splitting the faces is what happens behind you. Nothing clips triangles against the eye, so tri_persp simply refuses any triangle with a corner behind it, where z is negative. Looking straight at one wall from the middle of the cube, the walls either side of it each have two corners behind you, though part of them is in view at the edges of the screen. Drawn as two big triangles, they would vanish. Split into 16 squares, the only pieces that are lost are small ones off to the side.

From the centre of a cube, the walls never overlap on the screen either, so again there's no sorting or depth buffer, and the back-face test in tri_persp is commented out: every face is facing you.

Perspective-correct texture mapping

A simple texture mapper works out u and v at the corners of each line and steps them evenly across it. That's fine for a wall square on to you, but not for one at an angle. Say one end of a line of pixels is on the wall at z=1 with u=0, and the other end is at z=3 with u=256. The far half of the wall is squashed into less of the screen than the near half, so stepping u evenly puts the wrong part of the picture in the middle:

Half way across the screen, the right texel is 64, not 128. u itself doesn't change evenly across the screen, but u/z and 1/z do. So tri_persp steps those instead, and at each pixel divides one by the other to get u back: 42.7 / 0.667 = 64.

It starts by refusing triangles that reach behind the eye, and flipping u to 256-u, which turns each picture round the right way when it's seen from inside the cube. Then it sorts the corners from top to bottom, and swaps u, v and z for u/z, v/z and 1/z:

public void tri_persp(int texnum)
{
  if(tri_z1 < 0) return;
  if(tri_z2 < 0) return;
  if(tri_z3 < 0) return;

  int[] tex = this.tex[texnum];

  tri_u1 = 256-tri_u1;
  tri_u2 = 256-tri_u2;
  tri_u3 = 256-tri_u3;

  ...

  //- Divide by z to do perspective correction

  tri_u1 /= tri_z1; tri_u2 /= tri_z2; tri_u3 /= tri_z3;
  tri_v1 /= tri_z1; tri_v2 /= tri_z2; tri_v3 /= tri_z3;

  tri_z1 = 1.0f/tri_z1; tri_z2 = 1.0f/tri_z2; tri_z3 = 1.0f/tri_z3;

Gradients

Because u/z, v/z and 1/z all change evenly across the screen, each one is a flat plane over the triangle, and how much it changes for one pixel across, or one pixel down, is the same everywhere in the triangle. Those six numbers are worked out once, from the three corners:

float d = (tri_x3-tri_x1)*(tri_y2-tri_y1) - (tri_y3-tri_y1)*(tri_x2-tri_x1);

float dzx = ( (tri_y2-tri_y1)*(tri_z3-tri_z1) + (tri_y1-tri_y3)*(tri_z2-tri_z1) ) / d;
float dzy = ( (tri_x1-tri_x2)*(tri_z3-tri_z1) + (tri_x3-tri_x1)*(tri_z2-tri_z1) ) / d;

float dux = ( (tri_y2-tri_y1)*(tri_u3-tri_u1) + (tri_y1-tri_y3)*(tri_u2-tri_u1) ) / d;
float duy = ( (tri_x1-tri_x2)*(tri_u3-tri_u1) + (tri_x3-tri_x1)*(tri_u2-tri_u1) ) / d;

float dvx = ( (tri_y2-tri_y1)*(tri_v3-tri_v1) + (tri_y1-tri_y3)*(tri_v2-tri_v1) ) / d;
float dvy = ( (tri_x1-tri_x2)*(tri_v3-tri_v1) + (tri_x3-tri_x1)*(tri_v2-tri_v1) ) / d;

d is twice the area of the triangle on the screen, with a sign, and the rest is the plane through the three corners solved for its slopes. Here z means 1/z and u means u/z, since they were swapped over just before.

Filling the lines

The triangle is drawn in two halves, from the top corner to the middle one and from there to the bottom, like tri_col, but with the edges stepped down the screen by adding their slopes instead of being worked out again each line. side13onleft says whether the long edge, from corner 1 to corner 3, is on the left or the right. Rows and the ends of each line are rounded up to whole pixels and kept inside the clip rectangle.

Rather than following the left edge down, it keeps track of the values where each line would meet x=0, which only needs the slopes down. The start of each line is found from there, and then the slopes across are added for each pixel:

float zl = tri_z1 - tri_x1*dzx + (y1-tri_y1)*dzy;
float ul = tri_u1 - tri_x1*dux + (y1-tri_y1)*duy;
float vl = tri_v1 - tri_x1*dvx + (y1-tri_y1)*dvy;

for(j=y1; j<y2; j++)
{
  int xai = (int)xa; if(xai < xa) xai++;
  int xbi = (int)xb; if(xbi < xb) xbi++;

  if(xai < clip_x1) xai = clip_x1;
  if(xbi < clip_x1) xbi = clip_x1;
  if(xai > clip_x2) xai = clip_x2;
  if(xbi > clip_x2) xbi = clip_x2;

  float z = zl + xai*dzx;
  float u = ul + xai*dux;
  float v = vl + xai*dvx;

  int o = j*w+xai;

  for(i=xai; i<xbi; i++)
  {
    buf[o++] = tex[ ( (((int)(v/z))<<8) + ((int)(u/z)) ) & 0xFFFF ];//(((int)u)<<16)+(((int)v)<<8);
    z += dzx;
    u += dux;
    v += dvx;
  }

  xa += dxa; xb += dxb;

  zl += dzy;
  ul += duy;
  vl += dvy;
}

The inner loop is where the perspective correction happens: u/z there is (u/z) / (1/z), which is u. The texture is a flat array of 256 rows of 256, so the texel is v×256 + u, and the & 0xFFFF keeps it inside the 65,536 pixels when u or v reaches 256 at the edge. The comment after it is an alternative that draws the stepped u and v as red and green instead of the picture.

That's two divides for every pixel, which is the price of getting it exactly right. The bottom half is the same loop, with the short edge from corner 2 to 3 in place of the one from 1 to 2.

Unfolding the cube

The unfolded panorama is the same viewer drawn six times a frame. The 512x384 applet is cut into a 4x3 grid of 128x128 cells, and each of the six views is drawn into one cell, with the clip rectangle set to that cell and the centre of the projection moved to its middle:

The scale is set to w/8, 64 pixels, which is half a cell. A point at 45 degrees off to the side has x/z = 1, so it lands exactly on the edge of the cell, and each cell shows exactly 90 degrees. Six 90-degree views, one through each face, cover every direction once, so together the cells are a new set of six cube faces for the panorama as it's been turned. That's the texture being warped while the cube stays still.

Between views, the camera is turned a quarter turn with rotate, which applies after the mouse's rotation:

scale = w*1/8;
float hp = 3.14159f * 0.5f;

//  front

transformPoints(m);

clip_x1 = w*1/4; clip_y1 = h*1/3;
clip_x2 = w*2/4; clip_y2 = h*2/3;
cx = (clip_x1+clip_x2)>>1;
cy = (clip_y1+clip_y2)>>1;

for(i=0; i<numFaces; i++) drawSquare(i);

//  top

m.rotate(-hp, 0, 0);
transformPoints(m);

clip_x1 = w*1/4; clip_y1 = h*0/3;
clip_x2 = w*2/4; clip_y2 = h*1/3;
cx = (clip_x1+clip_x2)>>1;
cy = (clip_y1+clip_y2)>>1;

for(i=0; i<numFaces; i++) drawSquare(i);

The turns build on each other: up for the top, two quarter turns down for the bottom, back up, and then a quarter turn about y for each of left, back and right. All six faces of the cube are offered to every view, and tri_persp throws away what's behind it and clips the rest to the cell, so 1,152 triangles go in each frame and only the ones in view are drawn. The cells left over keep the background colour, 0x336699.

The logo

After every frame, a 210x18 logo is laid over the bottom left corner. It's made from two images: zn_mul, a soft dark shadow of the lettering on white, and zn_add, the lettering in white on black. Each pixel is darkened by the shadow, then by the lettering, and then the lettering is added on top:

r = (buf[o2] >> 16)&255;
g = (buf[o2] >> 8 )&255;
b = (buf[o2]      )&255;

zm = zn_mul[o1] & 255;
za = zn_add[o1] & 255;

r = ((r * zm)>>8);
g = ((g * zm)>>8);
b = ((b * zm)>>8);

r = ((za>>8) + (r)*(255-za))>>8;
g = ((za>>8) + (g)*(255-za))>>8;
b = ((za>>8) + (b)*(255-za))>>8;

...

buf[o2] = (r<<16)+(g<<8)+(b) + zn_add[o1];

Only the blue channel of each logo image is read, which is fine for grey images. za>>8 is always 0, so the lettering goes in with the last line, which adds the whole zn_add pixel. That pixel has FF in its top byte from PixelGrabber, which ends up in the top byte of the screen pixel, where the colour model ignores it.

The panorama viewer carries a Lewpen.com Panorama Viewer logo, and the unfolded panorama a Zero-Now Panorama v0.01 one.

Dragging

The mouse is read with the applet's mouseDown, mouseDrag and mouseUp methods. While you drag, each move adds how far the mouse went since the last one to two totals, and the main loop hands those to the scene and sets them back to zero:

while (true)
{
  scene.addRotation(mouse_changey, mouse_changex);
  mouse_changex = 0;
  mouse_changey = 0;
  scene.render();
  update(buf);

//      while(mouse_changex == 0 && mouse_changey == 0);
}

public void addRotation(float rx, float ry)
{
  drx -= rx*0.01;
  dry += ry*0.01;
  rchanged = true;
}

That's 0.01 radians a pixel, so dragging right across the 550-pixel applet turns the view 5.5 radians, most of the way round. Nothing stops the up and down angle at straight up or straight down, so you can keep dragging over the top and turn the world upside down.

render starts by checking rchanged, and returns without drawing if nothing has moved. But addRotation is called every time round the loop, and sets it whether the mouse moved or not, so every frame is drawn anyway. The commented-out line at the end of the loop was a wait for the mouse.

The spinning cube has the same mouse methods, which record where the mouse is, but nothing reads them.

Unfinished bits

• Before the photos are loaded, each texture is filled with a test pattern, (i^j) << (k*2), an XOR pattern in a different colour for each face. Loading the photo replaces the whole array, so the pattern is never seen.
• tri_col is still in the panorama viewers, unused, and so is tri_tex, a simpler texture mapper that steps u and v evenly. It never got as far as reading the texture: it draws a grey ramp from u, with the texture lookup commented out.
• tri_persp works out how 1/z, u/z and v/z change down each edge, as well as x, and only uses x.
• The wireframe flags and setWireframe, and the factex array, are never used. The u and v given to each point when the cube is made are marked "u & v not done yet!", and are overwritten when each face is drawn.
• In the unfolded panorama, the scale set when it starts, w*0.6f, is replaced by w*1/8 on every frame.

Running it here

The applets are the original class files, run by CheerpJ. A few things differ from their own pages:

• The spinning cube and the panorama viewer run at 550x400 here, rather than the 320x240 of their pages. They take their size from the applet, so the picture is just bigger, with the same view across.
• The photos are loaded from the folder the page is in, so the two panoramas run in small pages of their own, next to their JPEGs.
• The frame loops never sleep, so they keep a processor busy for as long as the page is open.