Behind The Scenes: How We Recreated Our Photobooks in 3D for Augmented Reality

Creating something you can be proud of is now easier than ever.

This month we launched a new feature in our iOS app which means you can flick through the pages of your creation as if it had already been printed.

Company Goals

When we started thinking about bringing 3D to our app we had three goals in mind:

1. Increase customer confidence

Allow customers to open their book design in 3D, turn the pages, and inspect details up close and from different angles.

2. Showcase premium features

Show customers the impact of different upgrades – such as UV gloss coating and hardback book covers – using their designs rather than just describing the benefits.

3. Catch design mistakes sooner

Give customers the opportunity to examine their design up close so mistakes can be caught before designs are printed and delivered.

Framework

The first decision at the start of any 3D project is which framework to use.

We looked at cross-platform engines like Unity, Filament and Babylon. Unity is a powerful engine but it’s more suited to games rather than as a component within an existing app.

Filament and Babylon are projects loosely sponsored by Google and Microsoft respectively, both have iOS libraries but they’re still in the early stage of development.

This left us considering the two 3D frameworks built into iOS, RealityKit and SceneKit. RealityKit is the newer framework built from the ground up to support AR experiences while SceneKit has been around for many years. We opted for SceneKit as it has a much broader range of features, for example you can write shader modifiers to deform geometry on the GPU in realtime.

Modelling

We offer three different kinds of book.

Our hardback and softcover books are perfect-bound while our photobooklets are spine stapled. Each binding type required a different approach and getting the hardback cover right needed a mini research project of its own.

Cover mechanics

If you study a hardback book cover up close, you'll find there's quite a lot going on mechanically.

The cover itself is made up of 3 pieces of card connected by a continuous paper wrap. This is then carefully glued to the perfect-bound pages inside. To model this in 3D, we first had to understand the mechanics.

Sketch analysis showing the trace of the book cover as it's opened:

The thin white lines in the tracing are our approximation of the different joints that give the book cover its flexibility.

Building this as a bone system in Blender allowed us to model the mechanics:

Perfect bound paper

Our hardback and softcover books both use "perfect bound" pages. This is where pages are bound by an adhesive strip along one edge.

As the book opens and pages are turned each page curves as different forces are applied to it. To approximate this in 3D, we needed to dust off our old school maths books:

This animated graph shows our mathematical model for perfect bound books.

This is a side-on view of a book, open at the middle spread, with a page being turned from one side to the other. A Bezier curve drives the curvature of the line with several dynamic parameters applied that try to mimic the way paper bends and bows.

Spine stapled paper

Our photobooklets consist of several long sheets of paper that are folded and stapled along the fold line.

The paper behaviour is similar to perfect bound books except that each page is half of a long sheet of folded paper. This is a subtle difference when viewed from a distance but to create an accurate model that stands up to close scrutiny we needed a new approach.

Multiple sheets of paper are wrapped around a central spine core to make up the booklet model.

A Bezier curve is applied to simulate bowing and flex during page-turning. Page textures are loaded in and out of memory efficiently as you move through the book.

Textures and materials

Generating page textures

Each page is generated dynamically based on the book's design.
This can take some time so as you move through the book we render and cache pages on-the-fly just ahead of when we need them.

We then juggle these page textures between disk, main memory and graphics memory to keep our memory usage under control.

Matte and gloss finish

We offer matte and gloss finishes for our photobooks. We calibrated our 3D materials to closely match the reflective qualities of the finished product.

This works even better in AR as the reflections take into account the environment around you thanks to ARKit's automatic environment texturing system.

Get in touch

We'd love to hear what you think about our new 3D Photobooks feature. If you've enjoyed this post and are interested in working on these kinds of problems please check out our jobs page:

https://popsa.com/careers/