It has been a long weekend, both at PennApps and for the Global Game Jam, with many thousands of participants – many of whom are strangers – coming together to work at a grueling pace, doing in hours what sometimes takes weeks, stressing as one experiments with new techniques and devices one may have never used before, cursing during the inevitable moments of frustration when things fail to work the first (or the second, or the third, or the n+1th) time, and either going with less sleep or in some cases, going with none at all.

Energy drinks, like Red Bull, in addition to the usual staples of coffee and soda (and something called Coffee Cereal, which as the name suggests, is a caffeinated, coffee-flavored cereal) are sometimes all that keep people going in the wee hours, with corporate sponsors often offering their wares to the weary ones, as seen below.

Of course, eventually, most people need to grab a few winks, but this it isn’t always in the most comfortable of places. At hackathons, people often have to…be as creative in finding places to lay their head as in working around limitations in code or hardware, sleeping on the floor, sprawled on a bench, on an air mattress or sleeping bag (for the fortunate ones), or sometimes just leaning back in their chair (as one of my teammates is doing below).

As you might recall from my last post, I live relatively close to PennApps, so I was fortunate enough to be able to go home and sleep in my own bed each night, but even that was only for a handful of hours, since there was a great deal of work to be done if were going to complete the gesture-based Internet of Things (IoT) control system we had in mind – and the model smarthome we needed to craft to demo it.

Now, many modern IoT rely on voice control, which as incidents with the Alexa and XBox One have demonstrated, are not always a great thing when our devices are left unsecured, given how much noise pollution we have in our homes and in our lives. And what security functionality we do have today is not exactly an improvement, given that one would have to speak the password every single time one addressed one’s device, making it both inconvenient to use, and a security problem in and of itself, since someone could overhear the password.

…that, or if someone manages to hack your device, which is linked to the web, they could potentially eavesdrop on you in your own home.

The solution our team decided upon was thus a local control hub that could turn off such features, utilizing a series of pre-programmed gestures to lock or unlock IoT devices, as well as allowing on to manipulate the features of one’s home via motion control (once unlocked). Since we didn’t have a ton of time to put this together, we used a Kinect (meant for the Xbox 360) and one of the Kinect4Windows libraries to test the proof of concept, and built out a little cardboard house with doors and lights the Kinect could be rigged to control.

After a good while of tinkering, fiddling with code – and interfacing Kinect, Arduino and other systems both out of necessity, since we needed those to control the model house’s functions, but also to test how interactions would work, we got the gesture lock/unlock and motion control scheme working.

It was a wonderful feeling – not least because, while using it, it was like using the Force to open and close doors, a sensation that felt almost magical, even though we knew exactly why and how it all worked). But beyond that, there was also the feeling of triumph, of overcoming obstacles and bringing something new into the world, after all the hours of stress, networking issues, Null Pointer Exceptions, and the like.

Yes, it had its issues – like the fact that more than one person in the field of view causes it to lock up and not respond to lock/unlock or input at all, but that was really more a feature than a bug. After all, its bad enough when people fight over a remote control: can you imagine what it would be like if people were fighting over opening and closing windows, dimming and brightening lights?

I can, all too well.

Anyway, we didn’t end up winning – the honor of first place at PennApps went to a group who developed Smart Tattoos, replicating some of the work that MIT Media Lab had done with “DuoSkin”, a method of rapid prototyping electronics on the human body by employing a variety of skin-friendly components. This group went further though, looking at how the technology could be used in the real world, adding a NRF52 microcontroller to eliminate the need for an external capacitive sensing chip, implementing Bluetooth Low Energy to relay data from the microcontroller to a computer or Android device with minimal power consumption and minimal work in connecting.

Basically, their team came up with a way of creating conductive temporary tattoos that could be used as control mechanisms on the surface of a human body, something which is really cool for someone like me who is interested in accessibility and different ways of interfacing with the world and the systems within it.

Smart Tattoos were just one of the cool ideas people came up with, and I hope they’ll go forward with it, as my team does with ours in some way.

You see, the thing about hackathons is that there’s so much potential there. Not just potential as in new technologies and new ideas waiting to be discovered, but new collaborations, new skills, new depths you may not realize you ever had. And if you are willing to go where the path may lead, or if need be, skilled and passionate enough to blaze a new trail altogether, you might be surprised by what you find.

Were there frustrations, issues, problems? Yeah there were. Sometimes we’d find the event program was incorrect, that room assignments were off, that the organized chaos of the event was teetering more towards chaos and less towards organization, but we got the job done.

We finished our hack, and it worked – that’s something to be proud of.

But even if you don’t, you learn, and in the learning, become something greater, so that next time you participate, next time you see a challenge in your work, you might go “huh” and think of things a different way.

So if you wonder why people might be willing to work on for hours and hours on end on either a hack of some kind, or a new game, to long hours, grueling trials and herculean, sometimes even Sisyphean-seeming tasks, its because hackathons and game jams are opportunities. They’re places where one can learn and experiment, where one can test one’s limits, push one’s boundaries, and maybe even surpass them, by trying something you might otherwise never have done. They’re places where one can be part of a bigger community all working in this atmosphere of creativity and optimism, of dogged determination and a resolve that no matter the problem, we can find a way.

That’s kind of what this sort of hacking is all about, and that’s why, in addition to being a nurse, game designer, and a few other things here and there, I’m proud to call myself a hacker.

Hey, I even got the T-shirt, penguin and everything.

No Linux distros were harmed in the creation of this shirt.

So in closing, here’s to everyone who was PennApps XV and made it such a great experience, with a particular shoutout to the Kinect Security team. And for those of you who weren’t there, perhaps I’ll be seeing you at PennApps XVI, on one of our streams, or a game jam near you!