Cannabis-infused drinks. Burgers grown in laboratories. Entire meals in bottles. Consumers, retailers and farmers alike are hungry for the next generation of food, and investors are beginning to acquire the taste, too. Early-stage investment in agrifood tech startups reached $10.1 billion in 2017, a 29 percent increase on the previous year.
Agrifood can be split into two parts. “Agritech” refers to technologies that target farmers. “Foodtech,” by contrast, targets manufacturers, retailers, restaurants and consumers. Jointly, the two have enough reach to impact every part of the production line, from farm to fork.
Recently, foodtech investments have led the charge, with Delivery Hero’s IPO and multi-million rounds in ele.me and Instacart. However, agritech deals are catching up: Indigo Agriculture and Ginkgo Bioworks raised $203 million and $275 million, respectively.
There’s also more acquisitions activity in the sector. Some recently baked news suggests that both Uber and Amazon could be in talks with Deliveroo for a potential acquisition. Meanwhile, John Deere put $305 million on the table for the robotics company Blue River Technology, and DuPont acquired farm management software Granular for $300 million.
So why the growing interest in agrifood?
Food is a huge market, and it’s changing fast
Back in 1958, there were 3 billion of us on the planet. Today, population size has reached 7.6 billion, and is due to hit a whooping 11.2 billion in 2100. That is a lot of mouths to feed.
But the appeal of the food market doesn’t stop at volume. Indeed, following Bennett’s Law, as people’s income increases their diet becomes more diverse. This economic compulsion to seek variety is being complemented by a rise in ethical consumers voting with their forks. Many have grown aware of the link between food and ecology, health and animal welfare. The number of vegans in the U.S. has increased six-fold in the last three years, and more than tripled in the U.K. over the past decade.
This is not simply a case of having our cake and eating it.
These dual trends have led to supermarket shelves and restaurant menus evolving at pace. Consumers are keen as mustard to find new and healthy “superfoods” such as insects — Eat Grub and Cricke — and new consumption-forms, from meal replacement options like Huel to vegan meal boxes such as allplants.
Combining the above two ingredients — a growing population and a diversification in diet — cooks up quite the appetizing dish for investors: The global food and agriculture industry is estimated to be worth at least $8 trillion.
New technologies are creating big opportunities
The food and agriculture value-chain is full of bottlenecks and inefficiencies. Some of them could be solved with the intelligent application of well-known technologies.
The humble online marketplace, for example. Marketplaces, including Yagro, Hectare Agritech and Farm-r, let farmers transact machinery and goods, while peer-to-peer platforms like WeFarm enable knowledge sharing. Food procurement marketplaces have cropped up too, such as COLLECTIVfood, Pesky Fish and COGZ — as have direct-to-consumer services, such as Farmdrop and Oddbox.
Some tech solutions are far more complex.
Genetic engineering, for one, is providing plenty of food for thought. Indeed, the UN suggests that food production must increase by 70 percent by 2050 to feed the world’s population growth. Genetic engineering could increase crop yields by 22 percent globally, as well as help pre-empt pre-harvest losses.
To this end, CRISPR is revolutionizing how food is grown. CRISPR technology helps producers optimize photosynthesis and the vitamin content of crops. Since it was first tested on tobacco production in 2013, CRISPR has been used on a range of crops, from wheat and rice to oranges and tomatoes; and for a whole spectrum of applications — from boosting crop resistance to pests, to improving nutritional contents. CRISPR is also being applied to livestock. At the Roslin Institute in Scotland, researchers have successfully used CRISPR to develop virus-resistant pigs.
In the same vein, there have been major advances in cellular agriculture. Cellular agriculture combines biotechnology with food and tissue engineering to produce agricultural products like meat or leather from cells cultured in a lab.
It is easy to see how cellular agriculture and the companies applying it, such as Meatable and Higher Steaks, could dramatically change farming and food production.
Therefore, investors have thus been tempted to take a bite at the “clean meat” industry. In Europe, Mosa Meat just raised $8.8 million, while U.S.-based Memphis Meats raised $17 million in 2017.
Even though products are yet to hit the shelves, the appeal is clear: The meat market will be worth $7.3 trillion by 2025, with a 73 percent increase in demand by 2050. And clean meat technology could allow for the production of meat at virtually infinite scale: In just two months, 50,000 tons of pork cells could be grown per bioreactor by using starter cells from 10 pigs. This could dramatically reduce the production cost of meat, and also its environmental cost: 6x less water is needed and 4x less greenhouse gas is emitted per pound of clean meat compared to “traditional” meat.
Artificial intelligence and machine learning is also impacting agriculture. One of the main opportunities, amongst many, is in precision agriculture.
Farmers now receive better information on crop status due to advances in image recognition, sensors, robotics and, of course, machine learning. Startups such as Hummingbird Technologies and Kisan Hub have developed solutions that outperform human “crop walks.” Similarly, Observe Technologies provides fish farmers with AI-powered insights to optimize feeding.
Consumer acceptance is also likely to be shaped by the economic, environmental and ethical implications of agrifood technologies.
Moving indoors, Xihelm (full disclosure, Oxford Capital is an investor) is developing a machine vision algorithm that enables roboticized indoor harvesting. Such technologies could help solve the labor crisis in agriculture: The 2017 labor shortage saw labor costs rise by between 9-12 percent in the U.K.
When the food moves from farm to retailers, supply chains can become unwieldy and difficult to manage. As a result, there is a $40 billion fraud problem in food. Blockchain technologies are being applied to solve this problem, powered by companies such as Provenance. Walmart recently announced that their leafy-green vegetable suppliers must upload their data to the blockchain, allowing them to trace food back to the source in 2.2 seconds instead of a week.
Agrifood tech is still an acquired taste
Although the agrifood market is huge and presents many opportunities for investment, it still isn’t quite the tech investor’s favorite dish. Yes, investment increased to $10.1 billion in 2017; however, fintech hit $39.4 billion in the same year.
There are several reasons. Digitization is growing, but it is slow. Farmers are understandably risk-averse. Their aversion is strengthened by the seasonality and fallibility of their activity. Most crops deliver produce once a year, so any missed harvest can have dramatic and long-lasting consequences. Implementing any large-scale technological solution represents a risk; therefore veering away from the status quo is a decision that cannot be taken lightly.
Regulation is a huge consideration for the sector. The Court of Justice of the European Union recently ruled that plants created with CRISPR must go through the same lengthy approval process as GMOs. In France in 2018, a law banned the use of terms like “meat” and “dairy” for vegetarian and vegan products — although it is not clear how this law will apply to cultured meat products in the future, nomenclature is a fight clean meat startups will want to win for the sake of consumer acceptance.
Consumer acceptance is also likely to be shaped by the economic, environmental and ethical implications of agrifood technologies. It is chastening to remember that agriculture employs one in four people in the global workforce, a large proportion of which are women.
The future of food could see unemployment issues in farming; large changes in livestock and feedstock production; and significant alterations in land management. Furthermore, gene editing is likely to benefit large corporations more than independent farmers — who could be put at risk.
This is not simply a case of having our cake and eating it. Instead, the ingredients need to be chosen with great care, or the “future of food” risks leaving a very bitter taste.
Despite a beat on its Q4 quarterly earnings, Apple shares still managed to take a beating Thursday.
Shares are down 7 percent in after-hours trading after the company released its Q4 quarterly earnings report, detailing $62.90 billion in revenue beating analyst expectations of $61.57 billion, with earnings per share hitting $2.91 beating an expected $2.78 EPS. The results represent a 20 percent year-over-year growth in revenues at the company.
The reason for the after-hours drop? Apple forecasted weaker than expected earnings for the holiday quarter. While analysts were expecting revenue guidance to hit $93 million, Apple forecasts between $89M and $93M with a midpoint of $91 million according to Reuters.
Apple shipped 46.89 million iPhones this quarter with unit sales staying flat but revenue jumping 29 percent, a result of Apple’s strategy this past year to hike prices of their most high-end devices. The average price of each unit was $793 versus $618 a year ago.
The company shipped 9.7 million iPads (a 6 percent decrease YoY with a 15 percent revenue decrease) and 5.3 million Macs (a 2 percent decrease YoY). Revenue on “Other Products” which includes Apple Watch, Apple TV, HomePod, AirPods and Beats headphones climbed 31 percent.
The company surprisingly announced on its investor call that in subsequent quarters it would stop breaking out unit sales of iPhone, iPad and Mac and would only report revenues. They will also be renaming “Other Products” to “Wearables, Home and Other Accessories”.
Beyond wrenching more money from users with hardware upgrades, Apple has continued the trend of pulling more revenue from user services like Apple Music, Apple Care and iCloud. The company reported that its Services division reached “an all-time-high of $10 billion in revenue” (well, actually $9.98 billion), climbing 17 percent year-over-year. That’s a slowdown in growth rates from last quarter where Services revenue climbed 31 percent year-over-year, though Apple notes this quarter’s numbers included a one-time accounting adjustment of $640 million.
Among the big geographical segments there was pretty unified growth in revenues. The Americas region jumped 19 percent, Europe popped 18 percent and Greater China went up 16 percent year-over year. Apple saw more substantial growth in Japan (34 percent).
It’s been a rough past few weeks for the Nasdaq, tech stocks have been floundering though Apple has weathered the storm far better than most on the heels of several new hardware announcements. Earlier this week, the company introduced new models of the iPad Pro, MacBook Air and Mac Mini at an event in New York. This comes on the heels of the release of three new iPhone models and a redesign of the Apple Watch.
Over the past several months, the company has been bumping the prices of its newest devices promoting a broader spread between their older releases and newest devices. The iPhone XS Max starts at $1,099, the Apple Watch Series 4 starts at $399, the new iPad Pro starts at $799 and the new MacBook Air starts at $1,199.
We’ll have more details later this afternoon during the company’s investor call.
Today, in San Francisco, at an inspriring gathering of roughly 400 women investors and founders organized by the months-old nonprofit AllRaise — a large and growing group of female funders and founders who aim to accelerate the success of their peers — we had the opportunity to interview Angela Duckworth. If her name sounds familiar, that’s because Duckworth, a psychology professor at the University of Pennsylvania, has gained game in recent years though a TED talk about grit that has now been viewed more than 15 million times, and a follow-up best-seller titled Grit: The Power of Passion and Perseverance.
Her biggest finding is that success has many features, including raw intelligence and adaptability, but the most crucial ingredient may be hard work over sustained periods of time, or grit. She also maintains that grit is not as fixed as are some other inherited traits, even while how gritty we are has a lot to do with our biology. In fact, Duckworth said at today’s event that she didn’t always think of herself as gritty. “When I was growing up in a bedroom community in South Jersey, not practicing piano when I should have been, I had a father who would literally out of the blue say things like, “You know, you’re no genius.” (After Duckworth was awarded a MacArthur Genius Fellowship in 2013, she laughingly told attendees, she “got to show up at his apartment 40 years later and say, ‘Actually, I am, officially. Officially, I am a genius.'”)
Given the nature of the audience, Duckworth spent much of her on-stage time talking about how VCs and founders might evaluate potential prospects — not by IQ scores or gender of other widely accepted predictors of success — but instead by trying to gauge the passion and perseverance of the people who come into their orbits. She also shared some ideas on how to “cultivate” grit. Because founders and investors alike might find some of her insights useful, we’re publishing them here, edited lightly for length.
TC: You believe that grit is far from immutable. Did you think of yourself as a gritty kid or did you become gritty once you figured out what you wanted to do with your life?
AD: I will say that if I look objectively at the data of my life, at what I was actually doing, that I absolutely evolved. My confidence increased, my leadership skills improved. I mean, I ran for some very low-level office in student government, like secretary publicity manager, and I couldn’t even land that as a high school student.
So I’ve evolved and increased in confidence. In particular, I have figured out what I want to do. It was until I was 32 and pregnant with my second child, actually, that I figured out that I wanted to be a psychologist. I started my PhD at that age. So I’d say. There is absolutely compelling evidence in neuroscience that people change dramatically, they tend to evolve, our interests sharpen, our confidence generally improves. The story of human nature is the story of plasticity.
TC: You say that you gained confidence over the years. Is that because you found something that you are so good at or because you were getting validation elsewhere that made the pursuit of this thing — psychology in your case — more meaningful to you?
AD: It’s complicated. There are many things that make you who you are. But there are two striking factors. One is relationships. So when you look at people who are very successful or resilient, positive human outcomes come from extremely stable, strong, loving relationships. It is it is probably the most striking predictor of positive outcomes. It helps with everything: cardiovascular disease, how long you’re going to live, your income, your wealth. Everything comes downs to having somebody who loves you. And though I did have an imperfect father, as many of us in this room did, I did feel loved. And at different points in my life, I felt cared for by other people who were not my parents, and it is really very important, including by the way, when you’re looking at building a company. I think that has to be part of the culture.
TC: Meaning a culture of professional development.
AD: Yes. If you don’t want to call if love, because that’s too cheesy, call it something else. But it needs to feature extremely positive social support. And long-term relationships need to be easy to create in that company, not difficult.
I think another secret to confidence is being able to point to small wins. Human confidence does not just come from pep talks with role models. There have to be small wins, even if they aren’t obvious wins. And if you want to cultivate successful entrepreneurs, there has to be a structure where they have smaller goals and enjoy victories over those goals.
TC: Presumably, most of the women in this audience, like you, have no shortage of grit or they probably would not be here. What I wonder about is when grit veers into workaholism. When is one working too hard?
AD: Yeah. The message of “Grit” is not that 90-hour workweeks [are ideal] or all-nighters because the emphasis on endurance suggests that you have to take care of yourself. I mean, if it were a 60-second sprint [that you were facing], I guess you could do all these unhealthy things. But it is a long-term goal that you’re working on, and therefore it’s incredibly important, the way you pick the way you want to do it. You have to care for yourself physically and emotionally to stay in the game.
Even Olympic athletes at the peak of their careers are doing probably four hours at most, pre-Olympics, of actual deliberate practice. So again, it should not be about sheer quantity but rather high-quality work and a sustainable routine.
TC: In addition to maybe working long versus working smart, I associate workaholism with needing or doing something for outside, versus internal, validation. Is that inaccurate?
AD: Data on this is also incredibly intuitive. Really successful entrepreneurs have enormous internal motivation. They aren’t waking up to do work because they ought to; they’re waking up because they want to [wake up and do their work].
TC: There’s a lot in your teachings about grit about stick-to-it·ive·ness. But in Silicon Valley, founders are known for pivoting when things aren’t working. When should people get credit for cutting their losses and moving on?
AD: I’ll give you my parenting advice for those of you with a kid in your life, because I think it’s also reasonably good VC advice.
In my family, everybody has a “hard thing” rule. Everyone has to pick a hard thing. It’s not “tiger” parenting. But you have to do something that requires deliberate practice, something that requires a cycle [wherein you’re] breaking things down into skills and then sub skills and really concentrating and getting feedback and doing it again.
Our second thing is that you can’t quit in the middle of something.
As a parent, I thought this was a way to teach work ethic, and teach our kids how to practice because that’s not something you are born knowing how to do. We wanted them to have some kind of bias to follow through on their commitments. It’s not a bad rule, I would imagine, for investing, too, or running a company. You want to help the people in your portfolio or the people in your company do things that are hard, to follow through on their commitments to their natural end while also making sure that that if you going to [stop this thing that you doing], that you decide it on a good day.
We all have a fight-or-flight system that gets activated on bad days, and it is generally not a good idea to use the front part of your brain if that has happened because these things are in conflict. But if on a really good day, after you’ve had your cup of coffee and you look and you’re like, I really don’t want to do this anymore, then that’s probably a rational decision.
TC: At Penn, you continue to study evidence-based ways to build character in kids. What can you tell this audience about tech’s impact on grit, for both children and adults?
AD: I’ve done a little bit of research on tech, but I can also summarize some of the research on social media in particular, and tech in general, and kids are for spending an unbelievable amount of time [with tech]. It’s about as much time as they are in school, actually, and equal number of hours.
When you look at longitudinal data to see what happens when people spend more time on social media, what happens in general is that people get more depressed and unhappy. It’s through his mechanism of comparison to others and envy. You know, there’s always a sunset and everyone looks beautiful and you think then that . you look like shit and you’re boring.
That’s the bad news. Here’s the good news. The sword cuts the other way as well. In many studies, when you use social media in an active way as opposed to passive way, so not scrolling through everybody’s pictures but actually communicating with other people and messaging your friends or even posting pictures of your own, it increases your feelings of social connection and well-being. So the sword is right now positioned the wrong way, against well-being and happiness and security, but it can cut the other way. And because tech isn’t going away, I think it’s the responsibility of this community to figure out how to make that happen.
A list of 50-plus companies, including some of tech’s top names, joined forces this week to pen a letter calling out the Trump administration over a reported plan to narrow gender definitions.
Apple, Google, Amazon, Facebook, Microsoft, Intel, Cisco and more drafted the letter (full text below) in response to a recent New York Times story about a planned federal rollback of Obama-era civil rights protections for transgender and gender non-conforming citizens. The move by the Trump administration set off a spate of protests around the world in support of transgender rights, and the response from the tech industry soon followed.
“We oppose any administrative and legislative efforts to erase transgender protections through reinterpretation of existing laws and regulations,” the note reads. “We also fundamentally oppose any policy or regulation that violates the privacy rights of those that identify as transgender, gender nonbinary, or intersex.”
Rights groups and activists are mobilizing against a reported Trump administration plan to narrowly define gender, a move that could dramatically reduce federal protections for and recognition of transgender people on October 28th, 2018 in Amsterdam, Netherlands. In Amsterdam people gathered Sunday night during a rally which was a call-to-action to all to members and allies of the trans, LGBQIA, black and brown resistance, immigrant and social justice movements to stand side by side with trans men, trans women and non-binary & intersex people, and to send a message of resistance and strengthen around the world. (Photo by Romy Arroyo Fernandez/NurPhoto via Getty Images)
The initial Times report stems from a memo proposing that the gender of individuals be solely based on their biological traits at birth.
“Proposed Definition: Sex means a person’s status as male or female based on immutable biological traits identifiable by or before birth,” the memo reads. “The sex listed on a person’s birth certificate, as originally issued, shall constitute definitive proof of a person’s sex unless rebutted by reliable genetic evidence.”
This isn’t the first time a Trump administration LGBTQ policy has united some of the industry’s biggest competitors. In February of last year, Apple and Google among others spoke out against the administration’s plans to roll back Obama-era guidelines surrounding transgender bathroom use in public schools.
Here is the full text of the letter:
We, the undersigned businesses, stand with the millions of people in America who identify as transgender, gender nonbinary or intersex, and call for all such people to be treated with the respect and dignity everyone deserves.
We oppose any administrative and legislative efforts to erase transgender protections through reinterpretation of existing laws and regulations. We also fundamentally oppose any policy or regulation that violates the privacy rights of those that identify as transgender, gender nonbinary, or intersex.
In the last two decades, dozens of federal courts have affirmed the rights and identities of transgender people. Cognizant of growing medical and scientific consensus, courts have recognized that policies that force people into a binary gender definition determined by birth anatomy fail to reflect the complex realities of gender identity and human biology.
Recognizing that diversity and inclusion are good for business, and that discrimination imposes enormous productivity costs (and exerts undue burdens), hundreds of companies, including the undersigned, have continued to expand inclusion for transgender people across corporate America.
Currently more than 80 percent of the Fortune 500 have clear gender identity protections; two-thirds have transgender-inclusive health care coverage; hundreds have LGBTQ+ and Allies business resource groups and internal training efforts.
Transgender people are our beloved family members and friends, and our valued team members. What harms transgender people harms our companies.
We call for respect and transparency in policymaking, and for equality under the law for transgender people.
Instagram hopes dollars from long-tail of small businesses and social media stars can help it pull its weight in the Facebook family. A new ad type called “Promote” for Stories allows Instagram business pages to show their ephemeral slideshows to more users without doing much work. Admins can choose to auto-target users similar to their followers, people in a certain location, or use all of Instagram’s targeting parameters to inject their Story into the Stories queue of more users as an ad that can also link to business’ Instagram profile or website.
Facebook confirms to TechCrunch that Promote for Stories works similarly to Facebook’s Boost option that lets them pay to instantly show their feed posts to more users. “I can confirm that we are testing this feature globally. We don’t have an immediate timeline for 100% rollout, but will keep you posted” an Instagram spokesperson told me. Screenshots of Promote were first shared by social consultant Matt Navarra.
Instagram already has 2 million active advertisers, compared to Facebook’s 6 million. But designing and targeting ads, especially full-screen video Stories ads can be daunting to small businesses and public figures. Promote offers an easy way to turn their existing Stories into ads.
The feature could unlock more spend at crucial time when Facebook’s revenue growth is in massive decline. It dropped from 59 percent year-over-years revenue growth in Q3 2016 to 49 percent in Q3 2017 to 33 percent in Q3 2018 as it hits saturation in lucrative developed countries and runs out of News Feed space. Facebook warned Wall Street about revenue deceleration as sharing shifts from feeds to Stories and advertisers have to adapt, but turning local merchants and influencers into paying customers could smooth that transition.
Instagram Analytics Launches In Beta
In other Instagram business news, today it launched Instagram Analytics in beta as part of Facebook Analytics. The tool goes beyond Instagram’s existing Insights tool that just counted different types of engagement with an account and its content, such as new followers, website clicks, post impressions, and Story exits. With Instagram Analytics, business accounts can track life time value and retention rates for people who do or don’t interact with their content, and create audience segments to see if people who commented on a particular post generate more value for them. They can also analyze how their Instagram audeince overlaps with people who visit their site, download their app, or Like their Facebook Page.
The more Instagram analytics businesses have access to, the better they’ll be able to prove that their investment in the platform is paying off. Being able to see exactly how followers move through a conversion funnel will result in higher confidence in campaigns and translate into more ad and content spend.
IGTV Hopes For Virality With Stories Previews
And there’s one final piece of Instagram news for the day. IGTV hasn’t quite blown up like Instagram Stories since launching in June, but a combination could bring some much needed attention to the app’s longer form video hub. Instagram today launched the ability to share preview image of an IGTV video to your Instagram Story. Friends can tap through to actually watch the full video on IGTV.
Now you can share your favorite IGTV videos to your story. Tap the paper airplane at the bottom of the video you want to share. When friends see your story, they can tap the preview to watch the whole video in IGTV. pic.twitter.com/oaatUoOqZY
IGTV has also recently added a History tab that shows what you’ve recently watched. This could be helpful for getting back to your favorite clips or jumping to a new episode of a show you’re hooked on.
Facebook CEO Mark Zuckerberg said on Tuesday’s earnings call that “People really want to watch a lot of video”, and the company plans to invest more in premium Facebook Watch content. But so far, it’s niether publicly announced any deals to pay for IGTV content, nor has opened any direct monetization options to creators. With viewership taking time to grow, there just aren’t enough incentives for creators to invest to producing polished, longer-form vertical video when there’s nowhere else to put it but IGTV. Virality through these previews could convince them there’s big fan base growth opportunities available if they stick with IGTV.
Combined, these updates show that the departure of Instagram’s co-founders hasn’t slowed down the company’s innovation. Former Facebook News Feed VP Adam Mosseri kept up a brisk pace of product launches, and now with Instagram he seems determined to keep users, creators, and businesses glued to what’s quickly becoming the social giant’s premier property.
Zume, the robotics and logistics company that got its start slinging out pizza, just raised $375 million from SoftBank, the WSJ first reported. SoftBank is also reportedly looking to invest an additional $375 million, which would value the company at a $2.25 billion valuation. The round comes a couple of months after reports of SoftBank looking to invest anywhere from $500 million to $750 million in Zume.
“We’ve recently closed a round of funding to support our growth and hiring,” Zume CEO Alex Garden said in a statement.
Zume, however, declined to comment on where the funding came from. Zume, which started back in 2015, uses robotics, artificial intelligence, automation and mobile kitchen technologies to predict food trends and serve up freshly cooked foods. The startup owns a patent for delivery trucks that can cook food while en route to customers.
Earlier this year, Zume created a larger umbrella company to house Zume Pizza, now a subsidiary of Zume. That marked Zume’s more ambitious approach to move beyond pizza and license its technology to restaurants looking to deploy food trucks.
“Pizza was our prototype,” Zume CEO Alex Garden told TC’s Brian Heater back in April. “There’s no reason why this technology wouldn’t work for any restaurant or any food category. Any restaurant who wants to adopt our system can now easily do that. They don’t have to be experts in technology or appliance manufacturing. They can just be restaurateurs, who have a more flexible offering for customers.”
Zume had previously raised about $70 million in funding. I’ve reached out to SoftBank and will update this story if I hear back.
From Elon’s Neuralink to Bryan Johnson’s Kernel, a new wave of businesses are specifically focusing on ways to access, read and write from the brain.
The holy grail lies in how to do that without invasive implants, and how to do it for a mass market.
One company aiming to do just that is New York-based CTRL-labs, who recently closed a $28 million Series B. The team, comprising over 12 PHDs, is decoding individual neurons and developing an electromyography-based armband that reads the nervous signals travelling from the brain to the fingers. These signals are then translated into desired intentions, enabling anything from thought-to-text to moving objects.
Scientists have known about electrical activity in the brain since Hans Berger first recorded it using an EEG in 1924, and the term “brain computer interface” (BCI) was coined as early as the 1970s by Jacques Vidal at UCLA. Since then most BCI applications have been tested in the military or medical realm. Although it’s still the early innings of neurotech commercialization, in recent years the pace of capital going in and company formation has picked up.
For a conversation with Flux I sat down with Thomas Reardon the CEO of CTRL-labs and discussed his journey to founding the company. Reardon explained why New York is the best place to build a machine learning based business right now and how he recruits top talent. He shares what developers can expect when the CTRL-kit ships in Q1 and explains how a brain control interface may well make the smartphone redundant. An excerpt is published below. Full transcript on Medium.
AMLG: I’m excited to have Thomas Reardon on the show today. He is the co-founder and CEO of CTRL-labs a company building the next generation of non-invasive neural computing here in Manhattan. He’s just cycled from uptown — thanks for coming down here to Chinatown. Reardon was previously the founder of a startup called Avegadro, which was acquired by Openwave. He also spent time at Microsoft where he was project lead on Internet Explorer. He’s one of the founders of the Worldwide Web Consortium, a body that has established many of the standards that still govern the Web, and he’s one of the architects of XML and CSS. Why don’t we get into your background, how you got to where you are today and why you’re the most excited to be doing what you’re doing right now.
TR: My background — well I’m a bit of an old man so this is a longer story. I have a commercial software background. I didn’t go to college when I was younger. I started a company at 19 years old and ended up at Microsoft back in 1990, so this was before the Windows revolution stormed the world. I spent 10 years at Microsoft. The biggest part of that was starting up the Internet Explorer project and then leading the internet architecture effort at Microsoft so that’s how I ended up working on things like CSS and XML, some of the web nerds out there should be deeply familiar with those terms. Then after doing another company that focused on the mobile Internet, Phone.com and Openwave, where I served as CTO, I got a bit tired of the Web. I got fatigued at the sense that the Web was growing up not to introduce any new technology experience or any new computer science to the world. It was just transferring bones from one grave to another. We were reinventing everything that had been invented in the 80s and early 90s and webifying it but we weren’t creating new experiences. I got profoundly turned off by the evolution of the Web and what we were doing to put it on mobile devices. We weren’t creating new value for people. We weren’t solving new human problems. We were solving corporate problems. We were trying to create new leverage for the entrenched companies.
So I left tech in 2003. Effectively retired. I decided to go and get a proper college education. I went and studied Greek and Latin and got a degree in classics. Along the way I started studying neuroscience and was fascinated by the biology of neurons. This led me to grad school and doing a Ph.D. which I split across Duke and Columbia. I’d woken up some time in like 2005 2006 and was reading an article in The New York Times. It was something about a cell and I scratched my head and said, we all hear that term we all talk about cells and cells in the body, but I have no idea what a cell really is. To the point where a New York Times article was too deep for me, and that almost embarrassed me and shocked me and led me down this path of studying biology in a deeper almost molecular way.
AMLG: So you were really in the heart of it all when you were working at Microsoft and building your startup. Now you are building this company in New York — we’ve got Columbia and NYU and there’s a lot of commercial industries — does that feel different for you, building a company here?
TR: Well let’s look at the kind of company we’re building. We’re building a company which is at its heart about machine learning. We’re in an era in which every startup tries to have a slide in their deck that says something about ML, but most of them are a joke in comparison. This is the place in the world to build a company that has machine learning at its core. Between Columbia and NYU and now Cornell Tech, and the unbelievably deep bench of machine learning talent embedded in the finance industry, we have more ML people at an elite level in New York than any place on earth. It’s dramatic. Our ability to recruit here is unparalleled. We beat the big five all the time. We’re now 42 people and half of them are Ph.D. scientists. For every single one of them we were competing against Google, Facebook, Apple.
AMLG: Presumably this is a more interesting problem for them to work on. If they want to go work at Goldman in AI they can do that for a couple of years, make some dollars and then come back and do the interesting stuff.
TR: They can make a bigger salary but they will work on something that nobody in the rest of the world will ever get to hear about. The reason why people don’t talk about all this ML talent here is when it’s embedded in finance you never get to hear about it. It’s all secret. Underneath the waters. The work we’re doing and this new generation of companies that have ML at their core — even a company like Spotify is, on the one hand fundamentally a licensing and copyright arbitrage company, but on the other hand what broke out for Spotify was their ML work. It was fundamental to the offer. That’s the kind of thing that’s happening in New York again and again now. There’s lots of companies — like a hardware company — that would be scary to build in New York. We have a significant hardware component to what we’re doing. It is hard to recruit A team world-class hardware folks in New York but we can get them. We recently hired the head of product from Peloton who formerly ran Makerbot.
AMLG: We support that and believe there’s a budding pool here. And I guess the third bench is neuro, which Columbia is very strong in.
Larry Abbott helped found the Center of Theoretical Neuroscience at Columbia
TR: Yes as is NYU. Neuroscience is in some sense the signature department at Columbia. The field breaks across two domains — the biological and the computational. Computational neuroscience is machine learning for real neurons, building operating computational models of how real neurons do their work. It’s the field that drives a lot of the breakthroughs in machine learning. We have these biologically inspired concepts in machine learning that come from computational neuroscience. Colombia has by far the top computational neuroscience group in the world and probably the top biological neuroscience group in the world. There are five Nobel Prize winners in the program and Larry Abbott the legend of theoretical neuroscience. It’s its an unbelievably deep bench.
AMLG: How do you recruit people that are smarter than you? This is a question that everyone listening wants to know.
Patrick Kaifosh, Thomas Reardon, Tim Machado the co-founders of CTRL-labs
TR: I’m not dumb but I’m not as smart as my co-founder and I’m not as smart as half of the scientific staff inside the company. I affectionately refer to my co-founder as a mutant. Patrick Kaifosh, who’s chief scientist. He is one of the smartest human beings I’ve ever known. Patrick is one of those generational people that can change our concept of what’s possible, and he does that in a first principles way. The recruiting part is to engage people in a way that lets them know that you’re going to take all the crap away that allows them to work on the hardest problems with the best people.
AMLG: I believe it and I’ve met some of them. So what was the conversation with Kaifosh and Tim when when you first sat down and decided to pursue the idea?
TR: So we were wrapping up our graduate studies, the three of us. We were looking at what it would be like to stay in academia and the bureaucracy involved in trying to be a working scientist in academia and writing grants. We were looking around at the young faculty members we saw at Columbia and thought, that doesn’t look like they’re having fun.
AMLG: When you were leaving Columbia it sounds like there wasn’t another company idea. Was it clear that this was the idea that you wanted to pursue at that time?
TR: What we knew is we wanted to do something collaborative. We did not think, let’s go build a brain machine interface. We don’t actually like that phrase, we like to call them neural interfaces. We didn’t think about neural interfaces at all. The second idea we had, an ingredient we put into the stew and started mixing up was, was that we wanted to leverage experimental technologies from neuroscience that hadn’t yet been commercialized. In some sense this was like when Genentech was starting in the mid 70s. We had found the crystal structure of DNA back in the late 40s, there had been 30 years of molecular biology, we figured out DNA then RNA then protein synthesis then ribosome. Thirty years of molecular biology but nobody had commercialized it yet. Then Genentech came along with this idea that we could make synthetic protein, that we could start to commercialize some of these core experimental techniques and do translation work and bring value back to humanity. It was all just sitting there on the shelf ready to be exploited.
We thought OK what are the technologies in neuroscience that we use at the bench that could be exploited? For instance spike sorting, the ability to listen with a single electrode to lots of neurons at the same time and see all the different electrical impulses and de-convolve them. You get this big noisy signal and you can see the individual neurons activity. So we started playing with that idea, lets harvest the last 30 or 40 years of bench experimental neuroscience. What are the techniques that were invented that we could harvest?
AMLG: We’ve been reading about these things and there’s been so much excitement about BMI but you haven’t really seen things in market things that people can hack around with. I don’t know why that gap hasn’t been filled. Does no one have the balls to go take these off the shelf and try and turn them into something or is it a timing question?
The brain has upper motor neurons in the cortex which map to lower motor neurons in the spinal cord, which send long axons down to contact the muscles. They release neurotransmitters that turn individual muscle fibres on and off. Motor units have 1:1 correspondence with motor neurons. When motor neurons fire in the spinal cord, an output signal from the brain, you get a direct response in the muscle. If those EMG signals can be decoded, then you can decode the zeros and ones of the nervous system — action potential
TR: Some of this is chutzpah and some of it is timing. The technologies that we are leveraging weren’t fully developed for how we’re using them. We had to do some invention since we started the company three years ago. But they were far enough along that you could imagine the gap and come up with a way to cross the gap. How could we, for instance, decode an individual neuron using a technology called electromyography. Electromyography has been around for probably over a century and that’s the ability to —
AMLG: Thats what we call EMG.
TR: EMG. Yes you can record the electrical activity of a muscle. EKG electrocardiography is basically EMG for the heart alone. You’re looking at the electrical activity of the heart muscles. We thought if you improve this legacy technology of EMG sufficiently, if you improve the signal to noise, you ought to be able to see the individual fibers of a muscle. If you know some neuroanatomy what you figure out is that the individual fibers correspond to individual neurons. And by listening to individual fibers we can now reconstruct the activity of individual neurons. That’s the root of a neural interface. The ability to listen to an individual neuron.
EEG toy “the Force Trainer”
AMLG: My family are Star Wars fans and we had a device one Christmas that we sat around playing with, the force trainer. If you put the device around your head and stare long enough the thing is supposed to move. Everything I’ve ever tried has been like that has been like that Force Trainer, a little frustrating —
TR: Thats EEG, electroencephalography. That’s when you put something on your skull and record the electrical activity. The waves of activity that happen in the cortex, in the outer part of your brain.
AMLG: And it doesn’t work well because the skull is too thick?
TR: There’s a bunch of reasons why it doesn’t work that well. The unfortunate thing is that when most people hear about it that’s one of the first things they think about like, oh well all my thinking is up here in the cortex right underneath my skull and that’s what you’re interfacing with. That is actually —
AMLG: A myth?
TR: Both a myth and the wrong approach. I’m going have to go deep on this one because it’s subtle but important. The first thing is let’s just talk about the signal qualities of EEG versus what we’re doing where we listen to individual neurons and do it without having to drill into your body or place an electrode inside of you. EEG is trying to listen to the activity of lots of neurons all at the same time tens of thousands hundreds of thousands of neurons and kind of get a sense of what the roar of those neurons is. I liken it to sitting outside of Giant Stadium with a microphone trying to listen to a conversation in Section 23 Row 4 seat 9. You can’t do it. At best you can tell is that one of the teams scored you hear the roar of the entire stadium. That’s basically what we have with EEG today. The ability to hear the roar. So for instance we say the easiest thing to decode with EMG is surprise. I could put a headset on you and tell if you’re surprised.
AMLG: That doesn’t seem too handy.
TR: Yup not much more than that. Turns out surprise is this global brain state and your entire brain lights up. In every animal that we do this in surprise looks the same — it’s a big global Christmas tree that lights up across the entire brain. But you can’t use that for control. And this cuts to the name of our company, CTRL-labs. I don’t just want to decode your state. I want to give you the ability to control things in the world in a way that feels magical. It feels like Star Wars. I want you to feel like the Star Wars Emperor. What we’re trying to do is give you control and a kind of control you’ve never experienced before.
The MYO armband by Canadian startup Thalmic Labs
AMLG: This is control over motion right? Maybe you can clarify — where I’ve seen other companies like MYO, which was an armband, it was really motion capture where people were capturing how you intended to gesture, rather than what you were thinking about?
TR: Yeah. In some sense we’re a successor to MYO (Thalmic Labs) — if Thalmic had been built by neuroscientists you would have ended up on the path that we’re on now.
Thomas Reardon demonstrating Myo control
We have two regimes of control, one we call Myo control and the other we call Neuro control. Myo control is our ability to decode what ultimately becomes your movements. The electrical input to your muscles that cause your muscles to contract, and then when you stop activating them they slowly relax. We can decode the electrical activity that goes into those muscles even before the movement has started and even before it ends and recapitulate that in a virtual way. Neuro control is something else. It’s kind of exotic and you have to try it to believe it. We can get to the level of the electrical activity of neurons — individual neurons — and train you rapidly on the order of seconds to control something. So imagine you’re playing a video game and you want to push a button to hop like you’re playing Sonic the Hedgehog. I can train you in seconds to turn on a single neuron in your spinal cord to control that little thing.
AMLG: When I came to visit your lab in 2016 the guy had his hand out here. I tried it — it was an asteroid field.
TR: Asteroids, the old Atari game.
Patrick Kaifosh playing Asteroids — example of Neuro Control [from CTRL-labs, late 2017]
AMLG: Classic. And you’re doing fruit ninja now too? It gets harder and harder.
TR: It does get harder and harder. So the idea here is that rather than moving you can just turn these neurons on and off and control something. Really there’s no muscle activity at that point you’re just activating individual neurons, they might release a little pulse, a little electrical chemical transmission to the muscle, but the muscle can’t respond at that level. What you find out is rather than using your neurons to control say your five fingers, you can use your neurons to control 30 virtual fingers without actually moving your hand at all.
AMLG: What does that mean for neuroplasticity. Do you have to imagine the third hand fourth hand fifth hand, or your tail like in Avatar?
TR: This is why I focus on the concept of control. We’re not trying to decode what you’re “thinking.” I don’t know what a thought is and there’s nobody in neuroscience who does know what a thought is. Nobody. We don’t know what consciousness is and we don’t know what thoughts are. They don’t exist in one part of the brain. Your brain is one cohesive organ and that includes your spinal cord all the way up. All of that embodies thought.
Inside Out (2015, Pixar). Great movie. Not how the brain, thoughts or consciousness work
AMLG: That’s a pretty crazy thought as thoughts go. I’m trying to mull that one over.
TR: It is. I want to pound that home. There’s not this one place. There’s not a little chair (to refer to Dan Dennett) there’s not like a chair in a movie theater inside your brain where the real you sits watching what’s happening and directing it. No, there’s just your overall brain and you’re in there somewhere across all of it. It’s that collection of neurons together that give you this sense of consciousness.
What we do with Neuro Control and with CTRL-kit the device that we’ve built is give you feedback. We show you by giving you direct feedback in real time, millisecond level feedback, how to train a neuron to go move say a cursor up and down, to go chase something or to jump over something. The way this works is that we engage your motor nervous system. Your brain has a natural output port — a USB port if you will — that generates output. In some sense this is sad for people, but I have to tell you your brain doesn’t do anything except turn muscles on and off. That’s the final output of the brain. When you’re generating speech when you’re blinking your eyes at me when you’re folding your hands and using your hands to talk to me when you’re moving around when you’re feeding yourself. Your brain is just turning muscles on and off. That’s it. There is nothing else. It does that via motor neurons. Most of those are in your spine. Those motor neurons, it’s not so much that they’re plastic — they’re adaptive. So motor control is this ability to use neurons for very adaptive tasks. Take a sip of water from that bottle right in front of you. Watch what you’re doing.
Intention capture — rather than going through devices to interact, CTRL-labs will take the electrical activity of the body and decode that directly, allowing us to use that high bandwidth information to interact with all output devices. [Watch Reardon’s full keynote at O’Reilly]
AMLG: Watch me spill it all over myself —
TR: You’re taking a sip. Everything you just did with that bottle you’ve never done that before. You’ve never done that task. In fact you just did a complicated thing, you actually put it around the microphone and had to use one hand then use the other hand to take the cap off the bottle. You did all of that without thinking. There was no cognitive load involved in that. That bottle is different than any other bottle, its slippery it’s got a certain temperature, the weight changes. Have you ever seen these robots try to pour water. It’s comical how difficult it is. You do it effortlessly, like you’re really good —
AMLG: Well I practiced a few times before we got here.
TR: Actually you did practice! The first year two years of your life. That’s all you were doing was practicing, to get ready for what you just did. Because when you’re born you can’t do that. You can’t control your hands you can’t control your body. You actually do something called motor babbling where you just shake your hands around and move your legs and wiggle your fingers and you’re trying to create a map inside your brain of how your body works and to gain control. But gain flexible, adaptive control.
AMLG: That’s the natural training that babies do, which is sort of what you’re doing in terms of decoding ?
TR: We are leveraging that same process you went through when you were a year to two years old to help you gain new skills that go beyond your muscles. So that was all about you learning how to control your muscles and do things. I want to emphasize what you did again is more complex than anything else you do. It’s more complex than language than math than social skills. Eight billion people on earth that have a functioning nervous system, every other one of them no matter what their IQ can do it really well. That’s the part of the brain that we’re interfacing with. That ability to adapt in real time to a task skillfully. That’s not plasticity in neuroscience. It’s adaptation.
AMLG: What does that mean in terms of the amount of decoding you’ve had to do. Because you’ve got a working demo. And I know that people have to train for their own individual use right?
Myo control attempts to understand what each of the 14 muscles in the arm are doing, then deconvolve the signal into individual channels that map out to muscles. If they can build an accurate online map CTRL-labs believes there is no reason to have a keyboard or mouse
TR: In Myo control it works for anybody right out of the box. With Neuro control it adjusts to you. In fact the model that’s built is custom to you, it wouldn’t work on anybody else it wouldn’t work on your twin. Because your twin would train it differently. DNA is not determinative of your nervous output. What you have to realize is we haven’t decoded the brain — there’s 15 billion neurons there. What we’ve done is created a very reduced but highly functional piece of hardware that listens to neurons in the spinal cord and gives you feedback that allows you to individually control those neurons.
When you think about the control that you exploit every day it’s built up of two kinds of things what we call continuous control — think of that as a joystick, left and right, and much left how much right. Those are continuous controls. Then we have discrete controls or symbols. Think of that as button pushing or typing. Every single control problem you face, and that’s what your day is filled with whether taking a sip of water walking down the street getting in a car driving a car. All of the control problems reduce to some combination of continuous control (swiping) and discrete control (button pushing.) We have this ability to get you to train these synthetic forms of up down left right dimensions if you will, that allows you to control things without moving but then allow you to move beyond the five fingers in your hand and get access to say 30 virtual fingers. What that opens up? Well think about everything you control.
AMLG: I’m picturing 30 virtual fingers right now —and I do want to get into VR, there’s lots of forms one can take in there. The surprising thing to me in terms of target uses and there’s so many uses you can imagine for this in early populations, was that you didn’t start the company for clinical populations or motor pathologies right? A lot of people have been working on bionics. I have a handicapped brother— I’ve been to his school and have seen the kids with all sorts of devices. They’re coming along, and obviously in the army they’ve been working on this. But you are not coming at it from that approach?
TR: Correct. We started the company almost ruthlessly focused on eight billion people. The market of eight billion. Not the market of a million or 10 million who have motor pathologies. In some sense this is the part that’s informed by my Microsoft time. So in the academy when you’re doing neuroscience research almost everybody focuses on pathologies, things that break in the nervous system and what we can do to help people and work around them. They’ll work on Parkinsons or Alzheimers or ALS for motor pathologies. What commercial companies get to do is bring new kinds of deep technology to mass markets, but which then feed back to clinical communities. By pushing and making this stuff work at scale across eight billion people, the problems that we have to solve will ultimately be the same problems that people who want to bring relief to people with motor pathologies need to solve. If you do it at scale lots of things fall out that wouldn’t have otherwise fallen out.
AMLG: It’s fascinating because you’re starting with we’re gonna go big. You’ve said you would like your devices, whether sold by you or by partners, to be on a million people within three or four years. A lot of things start in the realm of science but don’t get commercialized on a large scale. When you launched Explorer, at one point it had 95 percent market share so you’ve touched that many people before —
Internet Explorer browser market share, 2002–2016
TR: Yes and it’s addicting, when you’ve been able to put software into a billion plus hands. That’s the kind of scale that you want to work on and that’s the kind of impact that I want to have and the team wants to have.
AMLG: How do you get something like this to that scale?
TR: One user at a time. You pick segments in which there are serious problems to solve and proximal problems. You’ve talked about VR. We think we solve a key problem in virtual reality augmented reality mixed reality. These emerging, immersive computing paradigms. No immersive computing technology so far has won. There is no default. There’s no standard. Nobody’s pointing at any thing and saying “oh I can already see how that’s the one that’s going to win.” It’s not Oculus it’s not Microsoft Hololens it’s not Magic Leap. But the investment is still happening and we’re now years into this new round of virtual realities. The investment is happening because people still have a hunger for it. We know we want immersive computing to work. What’s not working? It’s kind of obvious. We designed all of these experiences to get data, images, sounds into you. The human input problem. These immersive technologies do breakthrough work to change human input. But they’ve done nothing so far to change human output. That’s where we come in. You can’t have a successful immersive computing platform without solving the human output problem of how do I control this? How do I express my intentions? How do I express language inside of virtual reality? Am I typing or am I not typing?
AMLG: Everyone’s doing the iPad right now. You go into VR and you’re holding a thing that’s mimicking the real world.
TR: What we call skeuomorphic experiences that mimic real life, and that’s terrible. The first developer kits for the Oculus Rift you know shipped with an Xbox controller. Oh my god is that dumb. There’s a myth that the only way to create a new technology is to make sure it has a deep bridge to the past. I call bullshit on that. We’ve been stuck in that model and it’s one of the diseases of the venture world, “we’re Uber for neurons” and it’s Uber for this or that.
AMLG: Well ironically people are afraid to take risks in venture. If you suddenly design a new way of communicating or doing human output it’s, “that’s pretty risky, it should look more like the last thing.”
TR: I’m deeply thankful to the firms that stepped up to fund us, Spark and Matrix and most recently Lux and Google Ventures. We’ve got venture folks who want to look around the bend and make a big bet on a big future.
Shouting “women’s rights are worker’s rights” and a number of other #TimesUp and #MeToo chants, upwards of 1,000 Google employees gathered in San Francisco’s Harry Bridges Plaza Thursday to protest the company’s handling of sexual harassment and misconduct cases.
Staffers from all of Google’s San Francisco offices were in attendance. An organizer, who declined to be named, told TechCrunch there were 1,500 Google employees across the globe that participated in the 48-hour effort to arrange a worldwide walkout. The effort was a major success. More than 3,000 Googlers and supporters of the movement attended the New York City walkout alone. As many as 1,000 Googlers and others came out for the San Francisco walkout, which the organizers said, was double the number they expected.
Cathay Bi, a Google employee in San Francisco and one of the walkout organizers, told a group of journalists at the rally that she was conflicted with participating in the walkout and ultimately decided not to go public with her own story of sexual harassment.
“I experienced sexual harassment at Google and I didn’t feel safe talking about it,” said Bi, pictured above. “That feeling of not being safe is why I’m out here today. I’d love it if everyone felt safe talking about it.”
“There were many times over the course of the last 24 hours that I emailed the group and said ‘I’m not doing this because I’m scared’ but that fear is something everyone else feels,” she said. “I said to myself last night, I hope I still have a career in Silicon Valley after this.”
Other organizers declined to go on the record.
There were protests around the globe today, including in London, Dublin, Montreal, Singapore, New York City, San Francisco and Cambridge, following a New York Times investigation that revealed Google had given Android co-creator Andy Rubin a $90 million exit package despite multiple relationships with other Google staffers and credible accusations of sexual misconduct made against him. That story, coupled with tech’s well-established issue of harassment and discrimination toward women and underrepresented minorities, was a catalyst for today’s rallies.
At the rally, Googlers read off their list of demands, which includes an end to forced arbitration in cases of harassment and discrimination, a commitment to end pay and opportunity inequity and a clear, inclusive process for reporting sexual misconduct safely and anonymously.
They’re also requesting that the search giant promote chief diversity officer Danielle Brown to a role in which she reports directly to chief executive officer Sundar Pichai, as well as the addition of an employee representative to the company’s board of directors.
Here’s the statement from Pichai Google provided to TechCrunch this morning: “Earlier this week, we let Googlers know that we are aware of the activities planned for today and that employees will have the support they need if they wish to participate. Employees have raised constructive ideas for how we can improve our policies and our processes going forward. We are taking in all their feedback so we can turn these ideas into action.”
Now, employees around the Globe will await Google’s highly-anticipated course of “action.”
“These types of changes don’t happen overnight,” Bi said. “If we expected them overnight we would have the wrong expectations of how these movements take place.”
Walmart is giving augmented reality a shot. The retailer today announced the launch of a new AR scanning tool in its iPhone application which will help customers with product comparisons. However, unlike a typical barcode scanner meant only to compare prices on one item at a time, Walmart’s AR scanner can be panned about across store shelves, offering details on pricing and customer ratings beneath the products it sees.
The technology was first developed by a team at an internal Walmart hackathon using Apple’s ARKit technology. At the time, their idea was to create a scanning experience that worked faster and felt faster when used by customers. They also wanted to build a scanner that offered more than just price comparisons.
“Walmart store shoppers love using our mobile app barcode scanner as a price checker. Our team sees the potential of this product as so much more, though,” explains Tim Sears, senior engineering manager at Walmart Labs, in a post announcing the feature’s launch. “When a customer launches the scanner, they get a direct connection between the digital and the physical world that their screen and camera lens creates for them,” he says.
The team won the hackathon, then went on to further redesign the experience to become the one that’s live today in Walmart’s application.
To use the scanner, you launch the feature in the Walmart app, then point it at the products on the shelf you want to compare. As you move the phone between one item and the other, the product tile at the bottom of the screen will update with information, including the product name, price and star rating across however many reviews it has received on Walmart.com. A link to related products is also available.
The AR scanner was designed to anchor dots to what you’ve scanned, but uses smaller dots instead of anchoring the entire content to the product itself to overcome the problems that could occur when multiple items are scanned together in a close space.
Despite the supposed advantages of AR scanning over a simpler barcode scan, it still remains to be seen to what extent consumers will adopt the feature now that it’s live.
Walmart isn’t the only retailer to give AR a go. Others have used it in various ways, including Amazon, Target, Wayfair and many more. But in several cases, AR’s adoption by retailers have been focused on visualizing products in your home, or — in the case of Target’s AR “studio” — makeup on your face.
Walmart’s AR scanner goes after a more practical use.
The AR Scanner is in the latest version of the Walmart iOS app (18.20 and higher), and works on iPhones that run at least iOS 11.3. This latter requirement is due to its use of ARKit 1.5, but will limit the audience largely to those with newer iPhones.
At least one million people will be receiving the next FabFitFun box as the Los Angeles company surpasses $200 million in revenue and continues its run as one of the startups to watch in the Los Angeles tech community.
As it renews its focus on media — doubling down on new programming in a bid to reach further into repeatable revenue through subscriptions that encompass more than just retail — the company is trying to frame itself as more than just makeup and accessories in a box.
“When we think of the potential behind the business … there are a few businesses in the world for whom membership is a no brainer. Netflix is, Spotify is and we think FabFitFun is a no brainer,” said Daniel Broukhim co-founder and co-chief executive of FabFitFun.
The company’s reach spans demographics and geography, according to co-chief executives (and brothers) Daniel and Michael Broukhim, with users ranging in age from 15 to 85 and subscriptions covering all 50 states.
“FabFitFun is truly for every woman – whether you are a millennial, a mom of three, or a fashion-forward 50 year old; we see this milestone as a celebration of the diversity of our members and that’s why we launched the #IamFabFitFun initiative,” said Katie Rosen Kitchens, the company’s other co-founder and editor in chief, in a statement. “We have members from all walks of life – from nurses to lawyers, software developers, police officers, makeup artists, fashion designers, dog walkers, interior designers and more.”
There will be a pop up with Macy’s department stores in the holiday season to merge the subscription business with brick-and-mortar retailers and the company is expanding further into health and wellness.
“We think about it in the context of a lifestyle,” said Michael Broukhim. “We’ve only been doing a low-level pilot on the fashion side. In the food space we’ve had snack vendors who have snacks in the main box. There’s a FabFitFun way to shop, [with] a discovery orientation. We do the heavy lifting for you and become storytellers in curating your life for you.”
The goal is to become a curator for more of its members’ interests, he said. “We want to do that for pretty much everything that someone consumes,” Michael said. “There’s the everything approach where you know what you want and you type it in and you get it and you click it and you get it delivered in maybe two days. Then there’s the FabFitFun approach…. it’s a trusted relationship where we learn about all the time and put as much of the process on autopilot as possible.”
With all of that, replenishment is not a focus for the company at the moment. “It’s upside for our brand partners,” said Daniel. “We’re helping their products get discovered. Then those members go to the brand partners and can continue on those relationships.”
