How to Navigate AI on the Edge (Device)

The pressure to innovate using AI at the edge in the restaurant, retail, and hospitality industries is building seriously fast. Virtual agents and robotics have escaped our scifi dreams and landed directly in our supermarket aisles, fast food drive-thrus, and hotel lobbies. AI is here, and flashy use cases are getting the headlines, but we all know that it’s going to take major infrastructure to flip that switch at scale, and that it won’t get built overnight (or, inexpensively).

If you’re a CTO or innovator tasked with developing an AI transformation strategy in one of these verticals, you already have an ear for the twin siren songs of “edge AI in a box” and the big hardware rip-and-replace. But you know a remix when you hear one: “Give us your wallet, and we’ll do the innovating!” Even if everything is changing, the business of selling change is as predictable as ever. And with budgets as tight as ever, we’re putting every vendor under a microscope. 

So, how do we get AI at the edge without breaking the bank?

New Edge AI, Old Edge Hardware: Flip, Don’t Rip

One answer we’ve been giving customers at Esper for years now is “flip, don’t rip.” Using their existing hardware, we’ve helped global brands (including one very famous purveyor of tacos) break out of dead ends and keep innovating on devices like legacy Windows point-of-sale systems. Systems that aren’t eligible to upgrade to Windows 11, typically, because the hardware is unsupported.

By flipping those POS devices to our custom Esper Foundation for Android x86 OS based on AOSP, we were able to keep them in action — but also make them faster, more secure, and responsive enough to meet the demands of a continuous innovation cycle. And the flip was remote and touchless, using a bootstrap file delivered to each flipped system over the network. As soon as the new OS booted, it would automatically self-configure once it was back on the network. Truly hands off. (Of course, we had a fallback for edge cases — a grab-and-go USB “flip stick” that would complete the process locally with the help of a technician.)

But you’re not going to just take my word for it — there’s got to be a lot more to it than that, right? Tons of pieces that just happened to fall the right way? Definitely, some things had to go right, and we’re not going to sit here and tell you that we’ll get your Ford Pinto running like a Bugatti. But it’s probably more approachable than you’d think. The big criteria we’re looking at in the feasibility of a Windows (or Linux) to Foundation (so, AOSP Android) flip aren’t anything exotic.

• CPU (something Intel in the last 10-15 years.)
• RAM / storage (Enough to run Android and store the OS + content.)
• Device suitability to purpose (Does it have a touchscreen? Do the peripherals work?)
• Device durability (No point in flipping what’s going to break next month!)

From there, it’s really about getting a POC built and offroading it in the lab — but given Android is a heck of a lot lighter and less demanding than Windows, we find old hardware gets some serious pep in its step post-flip.

Once you’re flipped, new possibilities emerge. You can deploy modern, cloud-driven AI experiences to get customers and employees the latest and greatest. And you can iterate on those experiences weekly, daily, or really, as often as you can sustain. It’s not delivering tacos on a robot, but it is a pathway to truly transformative CX.

On-Device Inferencing and Other Weird AI Tricks

Not every implementation of AI at the edge can lean on the cloud for the heavy lifting. While the cost of inferencing in the cloud is plummeting, the end-to-end latency of ingestion to output is simply too high for a lot of use cases like retail checkout, or in scenarios where internet connectivity is spotty or rate-limited. On-device AI to the rescue.

Every new SoC from every vendor has a massively capable NPU onboard these days, because on-device AI is table stakes for the consumer device segment. Enterprise and business get to ride those coattails with new capabilities in the latest and greatest hardware, but legacy devices are left out in the cold. Trying to run a modern computer vision inferencing model, for example, straight on the CPU — even a relatively fast, multithreaded desktop model — just doesn’t cut the mustard. You need dedicated silicon, be it an NPU, GPU, or something bespoke. So, do you justify an out-of-cycle rip-and-replace and write off otherwise perfectly good hardware just to enable a single capability? That’s a bitter pill to swallow.

With one of our customers, we found a middle ground. They need to verify the age of customers in a point-of-sale context, where near-real-time processing is essential. An ID needs to be scanned, verified, and a transaction cleared in a matter of seconds. Inferencing in the cloud wasn’t an option. But to completely relaunch with a new hardware platform to get dedicated silicon for on-device inferencing would have been massively expensive. They needed another option.

Instead, the customer developed their own edge compute solution: A small Linux-powered box that sits on-site, connected locally to every ID verification endpoint. The box has the dedicated hardware to do the inferencing, and the local network connection means ingestion to output time is nearly as good as running on the endpoints themselves. But each of those boxes needs a constantly-tuned version of the inferencing model, the latest OS and security updates, and to maintain tight compliance and high visibility at the management layer. Esper is what gives our customer the confidence in this solution — enabling that same constant innovation cycle, albeit to a different end, as our flip scenario.

Over the Edge: Physical AI and Beyond

Right now, we’re in the awkward teenage years of AI. We’ve learned to walk, and we know a lot of what we’re building toward, but those growth spurts of innovation come on suddenly, and we’re not always ready for them. We get caught flat-footed. But it’s clear that agentic experiences — and especially, robotics and physical automation — are going to be massive drivers of change in retail, food service, and hospitality. It’s already happening today; just think about where we’ll be in a decade (a taco delivery bot doesn’t sound so farfetched).

That future is going to require robust management at the edge, with physical machines constantly sharing data not just up to the cloud, but also with each other, as they refine their inferencing and use meshed communication networks to constantly optimize and reprioritize jobs. When combined with agentic customer experiences, businesses will be operating AI on-site at levels of sophistication and complexity that would have been unimaginable just 10 years ago. That brave new world is coming — and at Esper, we’re excited for it. But today, we want to help you through those gangly years of AI adolescence. There’s a lot left to learn, so let’s grow together.