Optimizing for Hololens and Low-End (video summary included)

“Tactical Twitch” is undergoing major refactoring (video summary at bottom). The current prototype works fairly solid, and displays well; however, it has trouble on low end devices. At this time, I am rewriting all of the functionality. This really sucks, but it will be completely worth it on the other end. Let’s look at what caused this situation, and all of the benefits a rewrite will include (hint: Hololens edition).

What Happened?

After my previous look at object pooling instead of instantiate and destroy, I began to realize a number of ways in which I’ve written unoptimized code. For one thing, object pooling is better done with Lists than Arrays. New insights such as this are coming from both peers and my nearly complete BS degree in software development from Bellevue University. I now have a much better grasp on good object oriented design.

The current “Tactical Twitch” demo represents about 2.5 months of development time, much of which occurred over two years ago. As I look back, I realize it is easier to rewrite the game than to optimize what exists.

I started this game in Javascript with enough self taught knowledge to accomplish just about any thing I could think of; however, everything I created struggles to run on low end devices. The addition of education has helped me recognize where my pitfalls are, and I there are bad practices threaded throughout my code.

At first this sounds like a terrible thing, but it doesn’t have to be. The prototype works well and to everyone I’ve shown it to personally, it has communicated the game I’m trying to make. It has received generally positive feedback from unbiased sources. I not only like what I’m making, but I have proof that it works and is fun. With that knowledge, I’ve left myself a clear blueprint of what it is I’m trying to code. It’s now simply a matter of execution. I look at the current build as a task list for the code I’m now writing.

I suspect there won’t be much visible progress until next Summer. I’ll likely start my next round of active development with something that looks very similar to what I have now. The difference will be that it will run on just about anything that plays Unity built games.

Benefits of a Rewrite

The new version is built with Hololens in mind. The same game will function well on low end devices, as well as deliver a slightly altered experience for Microsoft Hololens.

The new code has a nifty feature I’ve been working on, randomly generated levels. This is very important. As a one man shop it is hard enough developing a working game, let alone all of the content players will expect from it. Level design is largely what held back my “Legend of Sky” game. The platforming was received well with my select audience, but I just didn’t have time to create the levels. The algorithm I’m creating for the “Tactical Twitch” level creation can be reused in any other game I choose, and of any genre. To shorten that up, when complete, I will have largely removed my hurdle of time to develop content. Future games should be made faster.

The last minor (but still important) gameplay issues have be resolved via a slight re-design of gameplay. More on that when I reveal the new demo.

Video Summary

*Video note: My project files load slow in this video, because this is the second part of my GPD Win test. GPD Win is the first handheld Win 10 pc to be available. You can see my full review at techup.step2digital.com.

Thanks for reading and stay tuned folks. I look forward to giving you more updates and ultimately something to play!

8bit-ish | The Fair, Not the Renaissance

“With retro game art, players want a renaissance fair, not the actual Renaissance” – Mark Ferrari, #GDC16

Jump Strait to Resource Links

First, a Little Backstory

After 10 years of graphics and code, I’ve finally found a “professional hero”. Mark Ferrari is a veteran of the digital arts. He started at Lucas Arts where one of his larger contributions was “inventing” the concept of dithering in digital imagery. That’s right, there was a time when dithering wasn’t actually a thing. Mark is the guy who’s stroke of genius brought so much more life and perceived quality to the limited color pallets of early games. At the 2016 Game Developer’s Conference (#GDC16) I attended the session “8 Bit & ‘8 Bitish’ Graphics-Outside the Box”. Pixel art is something that I’ve never really tried, and so I am not very good at it; however, it is a style I’ve been wanting to experiment with for a while. Additionally, many of my students like this style. I chose this session in the hope that I would gain insight and tips to bring back to my class.

When I entered the session, I didn’t pay attention to who the speaker was. I was expecting a hot-shot young indie, and what I got instead was an industry veteran who helped pioneer quality game art in an era where “pixel art” wasn’t a style, but a technical limitation. In his talk, Mark details the process used “back in the day”, and his process for creating retro pixel art now. There is a lot of value in understanding how and why art was created in the fashion it was. That knowledge helps to understand what we’re trying to emulate, and what is worth emulating for modern pixel art. In this post I’ll highlight a couple interesting points on the history of game art, then I’ll move on to a couple of modern techniques/programs that Mark demonstrated the use of. The tidbits I gathered from this talk already have me creating better pixel art than any previous attempt I’ve made. I hope you’ll find some use for this information as well.

At the bottom of this article, I have compiled a list of links that should help to branch out and find more information on the topic.

EGA Graphics, EA Deluxe Paint, Palette Swapping | a.k.a “The Good ‘ol Days”

Enhanced Graphics Adapter (EGA)

EGA graphics in all of it's glory!
EGA graphics in all of it’s glory!

EGA was once the standard for color display on computer monitors. It offered 16 beautiful hardly palatable colors for display. Why were a particular 16 colors chosen for EGA? It turns out that the reason is because it was the job of programmers to select colors. At that time, colors had to be referenced by a numerical value. Programmers tend to like keeping things as simple as possible. The colors were chosen because they could be represented by integers, as appose to floats or doubles (decimal values).

The limited and “fuggly” color pallet is part of what helped Mark get started in game art design. Mr. Ferrari said that when Lucas Arts first approached him he felt he wasn’t right for the job, because he didn’t know much about computers. Lucas Arts said that was okay, because it’s much easier to teach an artist to work with a computer, than to teach a programmer to become an artist. Hiring Mark was a long term investment in improving the quality of visuals within the current technical limitations.

Deluxe Paint

One of Mark's old projects in Deluxe Paint, loaded inside of DosBox
One of Mark’s old projects in Deluxe Paint, loaded inside of DosBox

Once upon a time, Adobe was not the industry standard for digital imaging. In the prime days of DOS (early 90’s) it was Electronic Arts that offered the best tools for game art creation. The software they published was Deluxe Paint and Deluxe Paint Animation. In many ways the features of Deluxe Paint would still be better for creating low color/pixel graphics than Photoshop. Deluxe Paint paint is of course long dead. There are a number of reasons why DP isn’t a good choice for modern workflows, but there are specialized alternatives to Photoshop available. More on that later.

During Mark’s talk he used DosBox to run a copy of Deluxe Paint. Some of his original work that he used for illustration was only available in DP.

Palette Swapping

In addition to low color pallets, another technical limitation that once existed was the lack of transparency. There were no alpha channels. Graphics were all some form of a bitmap. In the time of EGA, Mark brought about higher quality visuals by introducing dithering to simulate the look of shadow, and depth-of-field. Eventually EGA graphics gave way to VGA, and there was now 256 colors to work with. This is where the dithering technique really began to shine, and it’s where Mark once again stretched the technical limitations with a technique to fake transparency called palette swapping.

Palette swapping is one of the most brilliant technical tricks I’ve seen an artist pull off. Here is how it works. When the color limitation increased from 16 to 256, certain amounts of color could be divided out for different uses. I don’t remember Mark’s exact numbers, so I’m just making them up for the purpose of illustration. An artist could choose work an image within less than 256 colors, such as 128, or 196, or whatever. The remaining color allotment is reserved for creating very subtle color changes in the image. These changes are so subtle that at a casual glance they won’t be noticed; however, when the color pallet in use with the image is swapped out for another, the new pallet has one or more of the subtle values changed to something much stronger. This is best seen to be understood.

Mark’s image that best shows this effect is his snow falling in the woods scene.trails If you look very closely at the image, you will notice vertical trails. The pixels in these trails are a slightly different color, but when the pallet shifts, they become white. When the pallet shift is in action, it looks like snow is falling, and one would believe there is transparency here, but it’s an illusion.

Some of Mark’s best palette shifting work has been preserved online by a developer who wrote an HTML5/JavaScript solution to perform the pallet swap. I highly recommend checking it out. There are amazing images where pallet shifting is used to simulate everything from waterfalls, to smoke and rain.

Flash Forward… How Do We Flash Backward?

Mark Ferrari has been contracted to create the artwork for the upcoming game “Thimbleweed Park”. The aesthetic goal of this game is to feel like an old school Lucas Arts point and click adventure. The key point here is that it should “feel” like the old games, not actually be an old game. That is an important distinction. The team believes that their players want to relive the feeling of those games; however, if one were to actually load one in something like DosBox, most individuals would be disgusted. What people love is the memory, or nostalgia of the old games, but when faced with the actual product, it’s typically a visual turn-off. This is especially true on modern high definition screens. When these old games were made the average computer monitor was a CRT Tube technology with 640 x 480 (VGA) resolution. Not only are the pixels big, but they bleed together. In that time, artists actually used the lower quality of displays to help convey visual effects in their artwork, and obscure necessary defects.

Mark eloquently put it like this, when people go to a renaissance fair they’re looking to have an enjoyable time. They want the costumes, the events, and the food. No one actually wants be in the Renaissance where crap flowed through the streets, death and disease were all around, the smell would have been unbearable to modern man. Folks just want to experience what they perceive to be enjoyable about the time period. It’s the same with modern retro games. People want a renaissance fair, not the actual Renaissance.

Here are some of the ideas Mr. Ferrari presented from his current workflow that help to inspire nostalgia, without the reality of old graphics.

Dithering

Dither and banding example one
Dither and banding example one

Let’s quickly define dithering, just in case we’re unfamiliar. Dither is the blending of colors through the application of noise between 2 or more colors. In a smooth transition, an image displays any number of colors creating a gradient. With dither, the colored pixels of two or more colors become intermingled. This intermingling of pixels is done through the application of noise patterns.

It is recommend to find or create ones own dithering patterns. This can be done in a number of ways, but the most precise is to create it oneself, pixel by pixel. It’s a lot of work up front, but once you’ve established your library of dithering patterns, future workflows will be fast and smooth.

You can visit this link to find out more about dithering patterns.

Color Banding

Dithering and banding example 2.
Dithering and banding example 2.

Color banding occurs one 2 colors have a hard transition, or no transition. A good example is a skyline. The sky may be light blue near the ground, and a deep blue higher up. if one only has 3 shades of blue to work with, the sky will have three bands going from light to dark. Dithering is used to intermingle the pixels along the edge, softening the transition. In older games that had limited color pallets, there was typically some amount of banding that could be noticed. While dithering eases the banding, too much dithering can result in a speckled mess of pixels.

Getting the best quality graphics in old games came down to balancing out color banding and dithering. Knowing that both would be noticeable, it wasn’t so much about eliminating them, as it was finding the balance that looked most ascetically appealing. This is where the eye of an artist was most needed.

Template Gradients

Create reusable dithering gradient patterns that can be applied quickly. As an example, Photoshop allows selections to be saved as fill patterns; so, create low index color gradients that will be used often, such as tree trunks (gradient going from the lit to the unlit side), and save as a pattern by selecting the low color gradient and choosing “edit > define pattern”, then use the pattern to fill areas that will become trees.

Understand Light in the Real World

Objects are not only lit by direct light sources, they are also lit by indirect sources. As an example, A tree is lit by the sun on one side; however, the opposite shadow casting side isn’t lit by the sun, nor is it black. It is illuminated by other objects that bounce light back. In this scenario, that would most commonly be the sky. When shading non directly lit portions of imagery, sample color values from other elements that would bounce light back. The background, or sky is typically a good source.

To assist in understanding where the indirect light is coming from, it is best to create backgrounds first. In doing this, one will have a better understanding of what colors to sample when highlighting and blending with foreground objects. This is the same concept as used in any painting art form. A few years ago I took some oil painting lessons. This was the first time I was introduced to the idea of blending foreground colors with the background. In traditional painting,  subjects one paints on top of the background will take on color properties of the background. This all assists in making an image look natural and appealing.

Limited (but not really) Color Pallets

At one time artists could only use 16 colors. Then they could only use 256. When creating retro art, enjoy the freedom of using how ever many colors are necessary to create something that is visually appealing; however, if creating something that is suppose feel nostalgic remember that a limited color pallet, or a at least the illusion of limitation is a needed component. In Mark’s session he mentioned that he wasn’t exactly sure how many colors were being used in each of the new “Thimbleweed Park” images,. One was around 60 and another was over 500.

My take away here is that a retro pixel artist can enjoy the freedom of using as many colors as necessary, with a mindset towards less. If one feels more colors are needed that is fine, but perhaps one should think about using more pronounced color banding and dithering to create the feel that there are less.

Decide for yourself what the limitation should be, and use that as your guide, not so much a hard and fast rule. Whether it’s modern or retro video game art, at the end of the day the goal is to create something that the players will want to look at. Again, make a renaissance fair, not the actual Renaissance .

Software Used by Mark in This Session

Photoshop – Adobe is the current standard, but there are some particular things that need to be given attention. Photoshop’s default setup is for modern high quality art. In order to create retro pixel art, keep in mind the following:

  • In preferences, make sure resizing is set to “nearest neighbor”.
  • Turn off all options for anti aliasing (depending on the tool in use, this is often found as a checkbox in the current tools settings)
  • Do your own anti aliasing to soften jagged edges where appropriate.

Following these guidelines should prevent Photoshop from trying to do it’s own re-sizing, and re-sampling of pixels (which looks nasty when size skew or enlarge).

Pro Motion by Cosmigo – Pro Motion is a dedicated pixel art tool. It contains many options geared towards animation, shading, and reuse/tiling of indexed and low pixel art. It is probably the best solution currently available, and the evolution of what EA’s Deluxe Paint would have become. Pro Motion, while not widely known in digital art, is used by some professional studios. Mark used it to create art for games such as Spyro : Eternal Night for GBA. More recently it has been used to create art for games such as Shovel Knight.

I’m not personally skilled in it; so I’ll link to some tutorials I found. I purchased a license after this session, and I’ll be getting to know it in the near(ish) future.

Conclusion

What I gathered from the session largely boils down to this; leave out the old limitations, and create your own. If one’s goal is to create something that feels retro, then use limited color pallets. Strategically place color banding and noticeable dithering to create a retro vibe, but don’t try to make an EGA game. Decide what you want, design limitations that will keep you focused on your visual goal, and hold loosely to those artificial limitations.

There was a lot to digest in this session. I’ve spent some time using the techniques Mark demonstrated, and I’m much happier with my results. I’ll post some of my new images as I finish them. Right now my pixel art is more of an experiment to study. The majority of my development time is going in to finishing the game I’m currently working on; however, you can bet some of this experience will make its way into my final art. There was more information given in this session. I’m hoping it will be in the GDC archive soon, I’d like to revisit it a couple of times.

Thanks for reading, and as promised here is that handy list of resources.

Tactial Twitch AR Prototype Website is Go!

* Maintenance Update: The Argon browser is currently at ver 4, and ver 3 is likely not on the app store anymore. This demo is not yet compatible with ver4. I will update it soon. Thanks for looking!

Just in time for the 2016 Game Developer Conference, the “Tactical Twitch” AR website is up for you to check out. The image target is my new business card. If you don’t have one, the image is on the instruction page, so you can print your own. At this time the AR browser is only on iOS, but is soon coming to Android. For all of the details, check out the video below, or head over to the new “hello” page.

Jurassic CART: Creating AR Experiences Catalyze Technical Education

Creating augmented reality games and experiences presents an amazing opportunity to entice our incoming adults to learn technical skills. I’ve long believed that game development is a necessary class to offer in the K-12 school system. Most students aren’t interested in subjects such as math and programming. Current programming courses tend to focus on business systems, and math is taught seemingly for the sake of… well, math. Ask most students if they want to build a database or solve equations, and the answer is unsurprisingly “not really”. Ask those same students if they want to make a T-Rex break through the wall of their school, and the answer is found in “Jurassic CART”.

Side Note: If you’d like to skip this article and check out the project page, go to jurassic.step2digital.com, and/or watch this news clip.

Story Time

As last Summer came to a close I had to shift my thoughts from my own game development pursuits, in favor of planning projects for my incoming class. I was about to begin my second year as the Interactive Game Design (IGD) Instructor for the CART High School in California’s Central Valley (yes, I mean Fresno).

File_000Every January the student’s attending CART’s various labs present a showcase project. Prior to my first year teaching the IGD course, I was an adviser to it; I knew that historically the game design presentation has been incomplete, and difficult for the public to understand. Now that I’m on the teaching end I’ve realized the students this course attracts typically do not have any previous experience making games, and they’re not necessarily the top academic performers. My student’s tend to be the ones with mediocre GPAs that are due in large part to playing League of Legends until 3am (the gamer in me is OK with this, the budding teacher is conflicted).

File_001It’s difficult to take a group of teenagers with a wide range of interests/ability and get them to a point where they can present a complete, understandable experience. This is where introducing augmented reality (AR) starts to make a lot of sense. The average consumer of digital media is not yet saturated with AR products. Even the simplest of AR demos, such as a floating cube is incredible to look at. I find that even I am captivated by the simplest of AR displays. It’s amazing to think that our digital worlds can overlay, and even interact with our physical surroundings.

The concept of AR is new to the consumer, and as a developer I’ve never messed with it before. I decided AR was the perfect project for this year’s showcase. The outcome does not need to be an immersive world with UI, inventory, or any of the staple systems a game would have. It just needs a bit of art that looks good, and requires just enough coding to run. It greatly simplifies what an entertaining end user experience could be for my class.

As an elective course, my IGD program in integrated with the student’s English requirement (taught by a separate qualified English teacher). One of the books or this year is Michael Crichton’s Jurassic Park. What a perfect pairing. What student (or teacher) wouldn’t want to bring dinosaurs back to life.

I’ll summarize the rest of this story with the following four points:

  • The project was a great success in the eyes of our audience, and in the entry level learning outcomes I was aiming for.
  • While the AR presentation works, it is still rough around the edges. The loading screen feels like one’s phone has froze (working this out is part of our continuing projects)
  • You can see our project in action in this new clip.
  • You can check out “Jurassic CART” for yourself at our official project page, jurassic.step2digital.com.

The Argon Project, and What’s Next

ArgonSiteIf you’d like to know more about what exactly my students did, and get ideas for projects you may want to do, check out the Argon project at Georgia Tech. Argon3 (currently only on iOS) is a web browser that can view AR content. I used this for our class project because developing content for it introduced students to basic programming concepts that surround web app development. Topics introduced include JSON, XML, HTML5 markup, and basic boolean logic/loops for making decisions.

As it stands, the web AR experience created by my students presented well to the general public that attended our showcase; however, it is still rough around the edges. First time loading often feels like the Argon app is crashing. We haven’t talked about the importance of loading screens yet. There are still a few months left in our year. I’m excited to see what we do next. We’re heading in the direction of level design, Unity3D, and possibly creating a Unity based AR game/experience. Whichever direction we go, my students are excited to see what they’re capable of. Though they still don’t recognize ( or like to acknowledge ) the amount of math they’re doing.