Innovation Transfer & Entrepreneurship
An Engineer, an Entomologist, and a Machine-learning Expert Create FarmSense to Improve Crop Yields and Reduce Food Insecurity
With a little help from their campus, three experts from UC Riverside have teamed up to create a smart insect monitoring system, FarmSense, to improve crop yields, reduce costs for farmers, and hopefully, reduce food insecurity and hunger. We sat down with Dr. Eamonn Keogh, professor of Computer Science at UC Riverside (and amateur insect-lover), for an insightful, fun discussion as he shared FarmSense’s journey through academia and into entrepreneurship.
Tell us a bit about your background and your co-founders.
I’m from Ireland…I grew up, went to college, got a PhD, and work in artificial intelligence and data-mining. Most data miners do the classic stuff – working for Facebook, Google, or in computational advertising – but I have no interest in people, or at least in social media stuff: most of my work involves animals or astronomy, or other nerdy stuff. I’ve worked with insects for 17 years and in doing so, met Agenor, a commercial entomologist. Over the years, we wrote some papers together, just for fun. Then six or seven years ago, we hit upon the idea of using sensors to identify insects in real-time. We’re so used to real-time these days – real-time traffic, real-time inventories – that we said ‘why can’t we have insect information in real-time?’ The answer is, there is no reason. So we played around with the idea academically for a few years, then about three years ago, took FarmSense on a more commercial track. Shailendra was a PhD student and post-doctorate who joined us as we co-founded FarmSense to help improve what we’d built (more on that later).
What problem is FarmSense tackling?
Insects destroy about $150 billion of food worldwide. Farmers have known about insect-related problems for thousands of years and use “interventions” as a means of trying to minimize crop damage caused by insects. Pesticides are a form of intervention, but there are also biological controls, such as releasing other insects into a particular area or using mechanical barriers in various forms. The key question the farmer is always asking is ‘what intervention should we do and when?’
To answer that question can be a bit tricky. Interventions can be expensive or cheap: perhaps you hire an airplane and spray your fields (expensive) or perhaps you buy some beneficial insects that prey on the pests and release them (much cheaper). It depends on what you have in your fields: what kinds of insects, what species, where they are, how many of them in a given location, etc. To make the matter more complicated, insects can be in clumps and thus only affect a particular area of a field. So as a farmer, it’s a constant challenge to know what you have to do to a field, or just a section of field, at any given moment.
The ever-nagging question of what intervention to do and when is what we’re tackling.
What are current solutions or interventions that farmers use?
We’re still quite Victorian when it comes to figuring out what interventions are needed. Farmers have sticky traps that resemble a stiff, yellow half sheet of paper covered in a glue-like substance and they put them out in their fields. Then 7-10 days later, they go back through all of their fields and look at or collect the sticky traps. They take the traps back to the farm for examination, identify what insects are on the traps, enter the data into a spreadsheet, then based on calculated thresholds and inputted data, the spreadsheet tells them what interventions are needed.
What are some of the major problems with sticky traps that FarmSense hopes to resolve?
The sticky-trap method is expensive and inaccurate. Man-power costs are expensive. Having up to 50 different types of insects on one trap and using the human eye to identify them yields inaccurate results. But the real kicker is that by the time the farmer realizes he has a problem, it’s already been 7-10 days, so he REALLY has a problem. Insect populations grow exponentially, so if you don’t catch them early enough, you have a really big problem. At that point, you have to call in a helicopter or an airplane to spray the crops, which costs a lot of money, as does the damage to the crops the farmer has probably already lost.
What exactly is your product? How does it work?
We’ve built a hollow, triangular-shaped sensor similar in size, shape, and color to the traditional sticky traps. Farmers place these sensors out in the field and never have to go back to them. An insect flies into the sensor (we attract them with appropriate baits or pheromones), the system identifies the insect immediately, pushes that information to the cloud, and the farmer can look at the information in real-time on our app. Within seconds our system is able to identify which insects are in your fields, not days.
We’ve spent the last few years optimizing the sensors and have a simple app that shows the data a farmer wants to know. We’ve included insect counts, broken down by species and sex (where appropriate), when they arrived, movements based on weather, etc. From there, a farmer can decide what interventions are necessary, where, and in a timely manner, reducing intervention costs and reducing crop destruction.
How does real-time insect identification actually work? How can a machine figure out which insect is which?
Inside the trap, there’s a tiny sensor that shines weak light onto photo transistors. When an insect flies in, its shadow causes the signal to change. The signal picks up the vibration of the insects wings and movement, all of which are optically recorded. The sensor can hear a tiny insect go past, but it wouldn’t hear a jack hammer or an explosion right next door – it’s pretty smart.
With the data that the signal picked up, it asks a machine learning algorithm to categorize the insect. For example, a typical honey bee beats its wings about two hundred times per second. So if an insect flies in and it’s not doing 200 beats per second, it’s highly unlikely that it’s a honey bee. But there are 18,000 species of honey bees. And there are 180,000 species of moths, 3,500 species of mosquitoes, and on it goes. So we have had to gather lot of data points about the species that we are interested in (we only really need to worry about commercially important pests).
Another variable the algorithm looks at are what time of day it is: moths like night and bees like the day-time. So the time of arrival gives you some information, add the wing frequency data, some other features and a bit of our “secret sauce” features, and the combo is put into our machine learning algorithm that then classifies the insect.
What gives you a competitive advantage?
We’ve been collecting data on insects over the last few years – tens of millions of insects. We breed insects, watch their entire life cycle at different temperatures and pressures, and basically have a model of bug behaviors and data points for millions of bugs. FarmSense has value in our sensors, in our algorithms, and definitely in our data. Our data is our best resource. It makes our classifications accurate, in most cases superhuman…that is to say, we can classify as well or better than the framer can.
We’re very fortunate being at UC Riverside and in entomology. The campus is known for its entomology, so when we tell entomologists around the world what we’re doing, they become very enthusiastic; so much so that we cut a deal with them: we’ll give you free sensors if you give us your data. We’ve essentially achieved volunteer crowdsourcing for insect data. People are using our sensors on four continents!
The data is what allows us to create such an accurate machine learning algorithm. Obviously, there is value in the impact FarmSense can have as well, but that’s not a competitive advantage.
What impact do you envision achieving with FarmSense?
We’d like to improve food security. Clearly, there’s a direct commercial application for companies such as Driscoll’s or Monsanto, but we’re also motivated by the small farm in Africa.
The corn earworm has moved into Africa and is causing incredible devastation on a continent that can’t easily afford it. We could change the profit margin of a big company, such as Monsanto, by a few percent, which is nice, but in Africa, we could make a difference to famine. That’s huge.
Having said that, directly helping people in Africa isn’t the best way to do a business. So our best bet is to commercialize in the developed world first then have it “trickle down” to the developing world.
Our impact goes beyond food security, though. For example, crops such as almonds frustrate farmers so much that often, they decide to spray them every week for, say, 12 weeks. If you had a FarmSense sensor that told you spraying was only needed 5 out of 12 weeks, then there’s significant cost savings already. But there’s more: chemicals cause environmental damage, so the less spraying farmers do, the less pesticides get into our food. There’s also the issue of pesticide resistance: insects develop resistance to pesticides. The more pesticides used, the sooner insects become resistant. Again, the less pesticides, the better. Producing and transporting said pesticides produces greenhouse gases, so the less, the better.
In short, when farmers have real-time data on insects, it reduces their expenses, and reduces the amount of pesticides used which is better for the environment and for humans. Also, now organic crops that couldn’t be made at a commercial level due to insect challenges can now be commercialized. So it’s a win-win-win.
What are the biggest challenges you’ve faced thus far and how have you overcome them?
Cost: there’s a magic price point for a sensor. Originally, when we first made one, the sensors cost around $500. The problem with that is thieves go through farms all the time and steal irrigation pipes or anything of value. They’d just steal the sensor if it was worth $500. Shailendra joined us because he had the perfect expertise for this issue with his background in sensors and embedded systems. I told him his challenge was to make our trap so cheap that it wasn’t worth picking up if you dropped it! Now it’s around $40 (and falling) and not worth stealing! Beyond theft, there is the pragmatic issue of deployment. For some crops/farms the farmer may need to deploy up to 100 sensors. Because we are so cheap now, we can afford to give the sensors away for free, and charge only a monthly subscription fee.
Power management was another challenge. Previously the sensors ran on motorcycle batteries, but now they’re running on tiny batteries which last an entire season. Shailendra did some very clever tricks with embedded systems and wireless hardware to reduce the energy consumption.
Another challenge were uncooperative insects. Several would land on the outside of the trap and walk inside, i.e. they wouldn’t fly past the sensor! How annoying! We need them to fly, so we had to incentivize them. We use a non-stick surface coating on the outside that essentially removes walking as an option. Then we also played around with the color, shape, texture, direction and anything else that affects the behavior of insects (Agenor’s specialty there).
When we put our sensors in fields in Hawaii, we started to get strange signals after a few days. It turned out that local geckos figured out that the insects inside were great to eat. The geckos would crawl inside and destroy the sensors. Of course – here’s the kicker - geckos are a protected species in Hawaii, so we had to figure out a harm-free solution to keep them out of our sensors! We coated the pole in a non-stick surface at the bottom – problem solved.
What resources within the UC system have been beneficial to you and why?
I’m not an entrepreneur at heart – we’re academics. We just brought in a new member to our team, Leslie Hicks, who has a PhD in entomology and has started and exited multiple businesses – she’s an entrepreneur at heart. We got connected to Leslie thanks to Riverside’s Chancellor of Research Michael Pazzani. Mike really believes researchers should be in entrepreneurs as well because it’s good for research, so he formed an advisory board that we as academics can talk to who are experts in their fields and it turned out that Leslie was on that board.
The Research and Economic Development team at Riverside are great. They are a one-stop resource for all your entrepreneurial needs: legal help, patent help, manufacturing help, reaching investors, pitching, getting feedback on pitches, 3D printing, everything. They helped us refine our pitch – us academics like to wind on, talk about the cool technical part, the algorithm, but investors want to hear about the business model. So they were really helpful.
Who have been key partners? Have you won any awards?
We’ve won awards from the Bill and Melinda Gates Foundation, an award from Vodafone Wireless Innovation Foundation, and received funding from NSF. We’ve also received money from Google and Microsoft as both are invested in insects, though only mosquitoes (they carry malaria and other human diseases).
Have you filed any patents?
We’ve got one patent so far and will file some more shortly. It’s funny, many things we’ve done don’t seem patentable, but patents are surprising: a new cell phone button might have 14 patents! We need to be more aggressive in our patents. Academic people aren’t as good at that, but I am learning.
What is FarmSense’s basic roadmap and what does the future hold?
We’ve spent the last two years refining our sensors to improve cost, robustness, and accuracy. Now it’s time to take off. We’ve done field tests with companies and those have been very successful. It took some test runs to convince farmers to have confidence in our traps, but after a few runs, we won them over. So traps and trials: done. Now it’s time to scale up.
We need to produce a lot more sensors and get some sales people. We’re looking to raise $1-2 million in the next few months and, fingers crossed, that should be fairly easy. Multiple investors have knocked on our door who heard of us through word-of-mouth or the academic world. Four of the five big agriculture companies knocked on our door with a lot of interest.
Next year we’ll probably raise another $5-10 million as it’ll probably be two years before we’re self-sustainable.
What advice would you give to fellow entrepreneurs in academia?
Get help – I know it’s obvious. When my students are failing or struggling, I tell them to get help and use resources available to them. For me as a professor, I had to do the same. Don’t do it alone, go get resources and get help. Especially in the UC system, there are people out there designed to help you as an entrepreneur.
The other would be you need to love what you do if you’re going to be an entrepreneur. As an academic, I work 40-50 hours a week but as an entrepreneur, you work 60-70 hours a week. The only reason I’d do that is because I love it. FarmSense is a blast. I love insects and solving problems, so if you can manage to make a product or company that involves your passion, it makes a huge difference to your motivation and work life.
Sign up for the UC Innovation and Entrepreneurship newsletter for more interviews with UC founders as well as opportunities to be included as a UC founder.