W3009: Integrated Systems Research and Development in Automation and Sensors for Sustainability of Specialty Crops
(Multistate Research Project)
W3009: Integrated Systems Research and Development in Automation and Sensors for Sustainability of Specialty Crops
Duration: 10/01/2018 to 09/30/2023
Statement of Issues and Justification
Increased demand for safe and high-quality food, pressure from global competition, and the need to preserve sustainability in natural resources and the environment, represent formidable challenges for specialty crop production in the U.S. Additionally, the decline in availability of skilled and unskilled farm labor is a continuing trend with very significant negative impact on specialty crop production. Producers and processors are urgently seeking new advancements in tools, methodologies, and systems that will aid them during production, harvesting, sorting, storing, processing, packaging, marketing, and transportation while maintaining competitiveness in production costs.
There is continuing need for sufficient and effective sensors, optimized assistive mechanization, and automation systems for specialty crops (fruits, vegetables, tree nuts, dried fruits and nursery). This is because many of the underlying biological processes related to quality and condition of fruits and vegetables are not readily transferred into engineering concepts. Biological variability, coupled with variable environmental factors, creates challenges and opportunities for developing sensors and automation systems for effective implementation at various stages of the production, harvest, and postharvest handling chain. Additionally, obtaining measurements of internal biological factors by using external, nondestructive sensors is not simple. Such devices or processes must adapt to a wide variation in shape, size, maturity, and other parameters of the commodity being processed. Although modern crop architectures, systems, and growing practices are slowly being upgraded to accommodate mechanized and automated production, a large portion of the production system, such as orchards, are still based on conventional, labor-intensive production practices. In contrast to grain crop production, it is a challenge for any single specialty crop sector to afford the cost of research, development, and commercialization of this complex level of automation, due to its relative small scale. It is thus important for public entities to assist this economically vital agricultural sector with integrated research and development in sensing, mechanization, and automation using an integrated approach.
Importance of the Work:
The steady increase in global competition and the recent decrease in available labor - especially skilled labor - have increased the need for new technologies. The specialty crop industry in the United States faces significant challenges to remain competitive; improving production efficiency is critical to keeping the specialty crop industry thriving. A system-wide approach to developing automation for the specialty crop industry is critically needed to address economic and environmental sustainability challenges. The proposed five-year project will address research and outreach needs for the specialty crop industry in these areas, working with researchers, manufacturers, and producers. Approval of this project will allow researchers and industrial partners to share their results and to plan future efforts in a coordinated manner to help develop new technologies to improve efficiency, to reduce redundancies, and help promote better use of expertise.
Related, Current and Previous Work
Specialty crop mechanization history
In the 1800's, U.S. agriculture was powered by human and animal labor. With the introduction of the steam engine and internal combustion engine, mechanical devices rapidly became the power source for agricultural operations. By the early 1900's there was a strong economic case for mechanical power (Ahndschin et al., 1921, Long 1931, Reynoldson 1933). The need for mechanization of agricultural crops (primarily grain and fiber crops) intensified after World War II (Cooper et al., 1947, Barlow and Fenske 1947, Lanham 1947). By the later half of the 20th century, grain and fiber agriculture employed very few farm laborers (National Rural Center 1980).
In contrast, specialty crops (fruits, nuts, vegetables, berries, nursery, dried fruits and herbs) continued to rely on human labor for most operations. Production operations were diverse, difficult, and required judgment and dexterity- attributes that were well suited for human labor and poorly suited for mechanization (Grise and Johnson 1973). There were a few examples of successful mechanization in production systems such as tree shakers (Pellerin et al., 1978), fruit conveyors (Rehkugler et al., 1976; Schulte et al., 1989) and orchard sprayers (Burton et al., 1989; Byers 1989, Giles et al., 1987). But most of the automation effort was in postharvest handling and processing. There were many research and development efforts in quality inspection of fruits such as apples (Brown et al., 1989; Brown et al., 1993; Cavalieri et al., 1998, Guyer et al., 1991), berries (Boyette 1996; Okamura et al., 1993; Timm et al., 1998) and other fruits and vegetables (leafy greens - Boyette et al., 1992; fruit - Crowe and Delwiche 1996; fruit handling - Deck and Strikeleather 1994; peaches - Miller and Delwiche 1989; cherries - Thompson et al., 1997; citrus - Miller and Drouillard 2001; fruit juices - Singh et al., 1996; prunes - Tan et al., 1990, and flowers - Gautz and Wong 1993).
US automation and mechanization research for specialty crop production all but ceased in the late 1980's and early 1990's. Research and development continued in Europe and Japan, outpacing the U.S. in production operations such as pruning, harvesting and handling specialty crops. In the late 1990's and continuing to the present, the farm labor supply for specialty crops has continued to shrink, and there emerged an urgent and on-going need for automation to replace human labor. The introduction of the USDA Specialty Crop Research Initiative (SCRI) program in 2008, along with support from NIFA through the AFRI and National Robotics Initiative programs have revitalized engineering-related research in the specialty crop area. Many projects have been funded that relate to this multi-state project and some have led to adoption of investigated technologies. For example, mechanized thinning of fruit has been proven to increase fruit size and quality in peaches, with significant reductions in labor required (Baugher et al., 2010, Miller et al., 2011, Schupp et al., 2012). By 2012, 60 "Darwin" and "PT-250" string thinners had been commercially sold in North America as a result of these studies. Also, the research knowledge and ecosystem created through such projects has contributed to a boom in agricultural mechanization and automation start-up companies, such as Blue River technologies, Abundant Robotics, and many others.
Assessment of plant and crop properties in the field
Significant effort has been performed to assess the status of plants using proximal and remote sensing technologies. Studies on citrus disease detection indicated that a promising future to manage HLB disease with ground-based and airborne hyperspectral sensors (Li et al., 2012a, b; Sankaran et al., 2011a, b) and/or fluorescence spectroscopy (Sankaran et al., 2012a) can be achieved. Similar hyperspectral technologies can be adapted to other specialty crops such as avocado (Sankaran et al., 2012b). Proximal leaf sensing was developed to sense water-stress in orchard trees (Dhillon et al., 2017). Also, unmanned aerial vehicles (UAVs) offer a wide range of advantages as sensing platforms in visual inspection systems. Senthilnath et al. (2016) developed a tomato detection system using spectral-spatial methods in RGB images captured by an UAV. Espinoza et al. (2017) used high-resolution multispectral and thermal images acquired from a UAV for water stress assessment in subsurface irrigated grapevines. Romero-Trigueros et al. (2017) analyzed structural and physiological changes of citrus crops caused by under water and saline stress using remotely sensed spectral data acquired from an UAV. Seyyedhasani et al. (2016) determined the accuracy level of Android-based mobile devices when used in agricultural fields for recording production data.
In addition to assessing plant status such as water and disease stress, an emerging area is high-throughput field phenotyping, i.e., the quantitative description of a plant's anatomical, ontogenetical, physiological and biochemical properties. These properties can be used for management and for breeding. Nguyen et al., (2015) used structured light and structure from motion and stereo (Nguyen et al., 2016) to perform 3D reconstruction for plants. Tang et al. (2015) developed a real-time crop stand analyzer for high-throughput infield crop sensing. Li and Tang (2017) developed a crop plant recognition algorithm to identify green bean and broccoli plants in weedy conditions by using 3D images. Bao and Tang (2016), Lu et al. (2017), and Li and Tang (2017) developed 3D computer vision algorithms to characterize the morphological traits of crop plants. Shah et al. (2016) developed a mobile robotic platform that was equipped with a suite of sensors to characterize different stress responses of plants growing in an array of growth chambers that simulate different growing environments. A robotic vision system, used for structural phenotyping and the first stage of a dormant pruning system, was described and its error assessed in Tabb and Medeiros (2017). Tabb (2013) used color cameras to reconstruct the shapes of thin agricultural objects such as fruit trees. Calibration error, common in outdoor situations, was mitigated in Tabb and Park (2015) when reconstructing tree structures. Existing methods for calibrating robots and cameras were not sufficient for reconstruction of thin structures; Tabb and Ahmad Yousef (2015, 2017) proposed new method for the robot-world, hand-eye calibration problem that resulted in reconstructions of trees with higher accuracy. This work was provided to the community via a data and code release (Tabb, 2017).
Machine vision systems will be an integral part of automated equipment, and significant progress has been made in using machine vision to detect, locate and assess yield and quality of fruits in canopies. In recent years, deep learning has also been used in vision systems. Slaughter and Harrell (1987) defined a color vision system for harvesting fruit. Cho et al. (2000) designed a real time tracking system in three dimensional space. Gongal et al., (2016) developed an over-the-row vision system for apple yield estimation, and Hung et al., (2015) utilized deep learning for automated yield estimation. Choi et al. (2016) developed a precise fruit inspection system for Huanglongbing and other common citrus defects in Florida using a graphical processing unit and deep learning algorithms. Gan et al., (2017) used vision to map immature citrus fruits in canopies. Choi et al. (2017) used a transfer learning technique to compare performance of RGB, depth, and near-infrared images in immature citrus detection system for yield forecasting. Chen et al. (2017) proposed a novel approach using deep learning to map input images to total fruit counts. Sa et al. (2016) studied a fruit detection algorithm using deep neural networks for various fruit types (sweet pepper, rock melon, apple, avocado, mango, and orange) in RGB and near-infrared images. Other sensing modalities have also been explored. For example, Cortes et al. (2017) integrated tactile sensing and visible and near-infrared reflectance spectroscopy in a robot gripper for mango quality assessment.
Assessment of crop attributes during postharvest operations
Postharvest needs to measure quality and inspect for damage have directed the physiological and metabolic study of specialty crops. Image analysis has formed the basis of many sensor designs. Machine vision system has been used to detect defects in prunes (Delwiche et al., 1990), peaches (Miller and Delwiche, 1991), and pistachio nuts (Pearson and Slaughter, 1996). Thai and Shewfelt (1990) monitored changes in peach quality during storage; Anderson and Abbott (1975) monitored apple quality in various storage environments. Vision systems have also been used to grade various fruits and vegetables; potatoes (Tao et al., 1995), apples (Rehkugler and Throop, 1986, Heinemann et al., 1994), roses (Steinmetz et al., 1994), mushrooms (Heinemann et al., 1994), raisins (Okamura et al., 1993), and citrus (Annamalai and Lee 2003, Okamoto et al. 2009, Kurtulmus et al. 2011, Bansal et al. 2011, Sengupta and Lee 2012, Yamakawa et al. 2012). Vision systems coupled with light energy transmitted through fruit has been used to detect watercore (Throop et al., 1994), and internal quality of mango fruit (Reyes et al., 2000). Durand-Petiteville et al. (2017) developed image processing algorithms to segment strawberry flesh and calyx. The parameters of firmness, color and bruise damage are recurring themes in reported work.
Firmness is associated with mechanical properties, and many researchers looked for connections between mechanical properties and sensory firmness. Watada et al. (1976) and Delwiche et al. (1987) tested devices to measure peach firmness. Davis and Pitts (1985) and Timm et al. (1996) developed portable devices to measure cherry firmness. Puncture tests as used by Timm and Guyer (1998), are the most common means to measure firmness, but are destructive. Nondestructive assessment of firmness is highly desirable, and multiple approaches have been studied to measure the mechanical properties in a nondestructive manner, such as impact loading (Finney et al., 1978), Reyes et al., 1996), Lu and Abbott, 1997) and vibration modal response (Wu et al., (1994), Lu and Abbott (1996)). A few indirect measures of firmness have been determined; near infrared reflectance (Lu et al., (2000)) and viscoelastic behavior (Gautz and Bhambare, 1990). Multiple investigators developed more universal fruit firmness sensors (Delwiche et al., 1987, Delwiche et al., 1996, Ozer et al., 1998, Hung et al., 1999).
Bruising is a common defect in fruits. Bruise detection is an important quality attribute for fruit and vegetables in terms of consumer acceptance and in storing the produce. Understanding the type and severity of mechanical loads which would cause bruising is important to the design of fruit and vegetable handling systems. Upchurch et al. (1987) developed a detection device based on ultrasound reflection. Hyde et al. (1990) used a combination of an instrumented sphere and camera to identify handling systems components that were likely to bruise fruit. Schulte et al. (1994) measured threshold levels of peach impact damage. Slaughter et al. (1993) quantified vibration injury to pears. Bajema et al. (2000) measured impact waves and resulting stresses in apple tissue. Rehkugler et al. (1971) and Upchurch (1991) used optical devices to locate bruises on fruit. Upchurch et al. (1990) developed a method to measure bruise volume in situ using spectrophotometers. Savary et al. (2011) studied the force distribution in citrus tree canopy during mechanical harvesting using a canopy shaker machine to improve fruit removal during harvest. Fan et al. (2017) identified optimum wavelengths for NIR hyperspectral reflectance imaging for blueberry internal bruising detection. Li et al. (2018) used hyperspectral imaging with improved watershed segmentation algorithm for detection of early bruises on peaches.
Color is one of the most important consumer quality attributes in fruits and vegetables. Color is also a maturity and harvest indicator. Color, both visible and infrared, has been the basis for numerous fruit and vegetable quality sensors. Delwiche et al. (1987) quantified color and maturity in peaches. Timm et al. (1993) made a similar harvest tool for tart cherries. Thai et al. (1990) monitored changes to tomatoes in storage. Kleynen et al. (2003) used selected frequencies of light to sort Jonagold apples. Infrared light has been used to measure firmness and sugar content (Lu, 2001), internal quality in peaches, nectarines and kiwifruit (Slaughter 1995 and Slaughter and Crisosta, 1998), and vegetable quality (Xie et al., 2007). Munera et al. (2017) used a VIS-NIR hyperspectral reflectance imaging to measure ripeness of two nectarine cultivars.
Some researchers have used novel methods to measure fruit quality. Pitts and Cavalieri (1988) used images of iodine stained starch locations in apples as a measure of maturity. Cho and Krutz (1989) and Ray et al. (1993) used NMR techniques. Timm et al. (1991) evaluated various technologies to detect pits in tart cherries. Zhao et al. (1993) used the apparent density of air-water mixtures to identity apples with watercore. Marrazzo et al. (2007) adapted an electronic nose design to differentiate between apple cultivars. Li et al. (2007) used sensor data fusion techniques to distinguish between damaged and healthy apples utilizing an electronic nose and a "zNose". Everard et al. (2017) developed a spectral imaging system that successfully detected fecal matter on all spinach leaf samples evaluated. Lefcourt and Siemens (2017) determined specifications for a low-cost imaging systems for detection of fecal contamination in produce fields.
Many other biological attributes have been studied. Sapers et al. (1977) measured to volatiles from a MacIntosh apple to estimate maturity. Delwiche and Baumgardner (1986) and Pitts et al. (1987) quantified the size of peaches and potatoes, respectively. Allison et al. (1987) measured internal pressures in maturing pecans. Watercore (a potential quality factor in apples) was studied by Hung et al. (1989) and Tollner et al. (1992). Pearson et al. (1996) studied hull adhesion forces in pistachio nuts. Ikediala et al. (2000) investigated dielectric properties of apple and codling moth larvae to develop a method to eradicate the insect from apples. Constante et al. (2016) used a deep learning architecture with noise compensated learning for strawberry classification in food processing industry.
Developing machine-friendly crop systems
Specialty crop mechanization is very challenging due to the differences between individual plants, such that model-based approaches (such as those used in factories) will fail (Fisher, 1992). Such 4-D architecture implies the absence of a consistent pattern that can be followed in finding limbs of trees or fruits/nuts on a limb and thus poses technological challenges to orchard mechanization. A conventional engineering solution to the problem is to develop special machines for each operation; however, the specialized equipment and small markets discourage innovation and commercialization by private sector technology providers. Therefore, mechanization solutions for tree fruit/nut production will be more successful if cropping systems are modified to accommodate the machinery rather than designing the machinery to fit all situations. Robinson et al. (1990) discussed the potential benefits of pruning trees to ease automated harvest operations. Schupp and Baugher (2011) and Miller et al. (2011) showed that narrowing canopy widths to make them more two-dimensional has increased the viability of mechanized fruit thinning. New training system designs are described in Baugher et al. (2003) and Crassweller and Smith (2013). Castillo-Ruiz et al. (2017) analyzed different tree structures in three pruning treatments to compare the performance of two mechanical olive harvesters. Schupp et al. (2017) developed a pruning severity index (limb to trunk ratio - LTR) for apples trees calculated from the sum of the cross-sectional area of all branches on a tree at 2.5 cm from their union to the central leader divided by the cross-sectional area of its central leader at 30 cm from the graft union. The LTR provides a measurable way to define and create different levels of pruning severity and achieve consistent outcomes, allows a greater degree of accuracy and precision to dormant pruning of tall spindle apple trees. This metric is also a necessary step in the development of autonomous pruning systems. Recent studies aimed at providing data and methods to enable model-based design of fruit-picking robotic arms. Arikapudi et al. (2016) developed technology to measure the spatial distributions of fruits in tree canopies and Vougioukas et al. (2016) calculated fruit linear reachability in high-density orchards. Bloch, Degani and Bechar (2018) used tree growth modeling software to co-optimize robotic manipulators and tree structure (Y-trellis branch angles) to maximize fruit picking efficiency.
Design of automated and mechanized systems
Major areas of mechanization and automation research include harvesting, weeding, thinning, fertilizing, planting and transplanting, robot mobility and robot-crop interaction, pesticide application and irrigation.
Mechanized harvesting is a major goal for specialty crop production. Harvesting is one of the most labor-intensive operations and has a high timeliness cost associated with delay (Oliveira et al., 1993). One approach is harvest-assist technology that bridges the gap between fully manual harvest and fully automated harvest. Such units for apple harvesting were developed by researchers (Schupp et al., 2011, Lewis et al. 2012, Heinemann et al., 2012). These units mount on a platform and utilize vacuums to transport apples from the picker through tubes to the apple bin. Results have shown improved picker efficiency and reduced potential for injury. Vougioukas et al., (2012) developed the concept of teams of small robotic strawberry transport carts acting as strawberry harvest-aid. Zhang et al. (2016) developed an economical harvest-assist device that mounts on a two-person picking platform, increasing harvest efficiency by 78% through elimination of ladders and collection buckets. Ye et al (2017) developed bin-carrying harvest-aids for orchard operations. Robotic harvest-aids can reduce non productive walking times during berry harvesting, and Khosro Anjom et al. (2018) developed algorithms to predict the picking time of strawberry pickers so that robotic harvest-aids can be pro-actively dispatched to them. The ultimate goal is fully mechanized harvesting. Recent work has re-visited the catch-and-shake concept. Larbi et al. (2015) improved a two-unit sweet cherry harvester using a continuous shaking mechanism in commercial operation. He et al. (2017) developed a new Shake-and-Catch harvester (shaking targeted tree limbs and catching the detached fruits) for fresh market apples in Trellis-trained apple trees. Robotic selective harvesting has been a research goal for many years. Sarig (1993) provided a state-of-review in the mid 1990's. Arima et al. (1996) developed a cucumber harvester in Japan. Kondo et al. (1996) developed a guided robotic system for cherry tomato harvesting. Pilarski et al. (2002) described a harvesting system using the Demeter system. Recently, Silwal et al. (2017) developed a low cost robotic apple harvester with sensing, planning, and seven-degree freedom manipulation functionalities in a modern orchard system with a planar canopy.
Automated weed control applications for Western cropping systems were reviewed by Fennimore et al. (2013) and field applications were reported. Fennimore et al. (2016) reviewed technologies for automated weed control and concluded that robotic weeding systems show great promise for use in specialty crops. Siemens (2016) reported on the advances and current state of robotic weeding systems for specialty crops. Siemens and Gayler (2016) developed design modifications for vegetable planters that significantly improved seed spacing uniformity. Lati et al. (2016) evaluated a commercial, automated in-row weeding machine in lettuce and broccoli and found the device reduced weed densities by 27 to 41% more than the standard cultivator and lowered hand-weeding times by 29 to 45%.
Mechanized thinning of fruit has been proven to increase fruit size and quality in peaches, with significant reductions in labor required (Baugher et al., 2010, Miller et al., 2011, Schupp et al., 2012). Kon et al., (2013) determined optimum string thinner spindle speeds for mechanical thinning of apple blossoms based on thinning response and minimized injury to spur leaves. They determined that mechanical string thinning might be a viable treatment in organic apple production, where use of chemical thinners is limited. Lyons et al., (2015) developed robotic peach blossom thinning algorithms based on heuristics.
Point injection fertilizer application was investigated in a three-year study in iceberg lettuce by Siemens and Gayler (2016). They found that use of an alternative, applicator that placed fertilizer in the root zone increased nutrient uptake efficiency by over 20% and crop yield by 19% when deficient rates of nitrogen were applied.
Planting and transplanting operations have been studied by a number of researchers. Especially in nursery operations, reducing labor used for transplanting is a critical need. Kutz et al. (1987) described a method for transplanting bedding plants. Gautz and Wong (1993) described a system to open flower pollen vessels during micropropagation. Kondo et al. (1996) described a vision system for cutting chrysanthemums. Sakaute (1996) developed an automated transplanter in Japan. Wang et al. (1997) developed a vision-guided robotic system for separating and transplanting sugarcane shoots from tissue culture.
Autonomous mobility is required for efficient and safe deployment of automated systems in orchards. Torii (2000) described the development of autonomous vehicles in Japan; Stentz et al. (2002) described a semi-autonomous vehicle developed in the US. Guidance is a critical part of vehicle mobility (Reid at al., 2000). A number of researchers have investigated ways to map the route of an autonomous vehicle using GPS (Bell et al., 1998, Freeland et al., 2002) and remote mapping (Tao and Li, 2007). Other systems rely on machine vision to move in the orchard (Slaughter et al., 1999, Pinto et al., 2000). Bergerman et al. (2012) developed autonomous orchard platforms to perform multiple tasks, including scouting, spraying, mowing, etc. These units were extensively field-tested in commercial orchards. However, they relied on artificial landmarks for turning between rows. Ye et al., (2016) developed steering control strategies for a four-wheel-independent-steering bin managing robot. At a mission-planning level, Bochtis et al. (2015) developed route-planning algorithms for autonomous vehicles in orchards, and Vougioukas et al. (2016) developed a safety system for detecting and localizing workers in orchards to enable safe operation of autonomous vehicles. Seyyedhasani and Dvorak (2017) developed a method that enables optimizing the routing of vehicle fleets in specialty crop production fields. Rounsaville, Dvorak and Stombaugh (2016) proposed improved methods of calculating the travel accuracy of self-driving vehicles so that performance is better described and Specialty Crop producers (for whom this accuracy is critical given the high value of crops) can better assess vehicle suitability in their applications. Jackson and Dvorak (2016) developed a small diesel-electric hybrid drivetrain (~20 hp) to enable high efficiency and long operating time robotic applications in specialty crops. Dvorak, Stone, and Self (2016) developed sensors to detect human presence from agricultural machinery, which is always a concern when humans work in close proximity to machinery as is common in specialty crop production.
Physical robot-crop interaction such as grasping, cutting, etc., is essential to many production operations. Sivaraman and Burks (2006) developed performance indices for manipulators. Cho et al. (2002) adapted three degree of freedom robotic arms for cutting lettuce heads. Edan et al. (1992) developed a finite element model of a gripper. There have been many designs for grippers of fruit in general (Kondo et al., 1992), melons (Cardenas-Weber et al., 1991), tomatoes (Kondo et al., 1995), and in the vineyard (Monta et al., 1995), but there is not wide acceptance of any particular design. A common gripper design would greatly advance the development of harvesting equipment.
Precision pesticide application has also been a topic of active research. Khot et al. (2012a, b) retrofitted an axial-fan airblast sprayer for use in citrus with adjustable air-assistance and liquid flow rates. Spray patterns evaluation and resulting spray decision rules formulated to operate the retrofitted sprayer were effectively tested under field conditions in varied sized citrus canopies. The results revealed that variable rate spray applications would result in 50% or less chemical usage while having comparable spray deposition to that of control. Oberti et al. (2016) implemented selective targeting of pesticide applications in grapevines for disease control using a modular agricultural robot.
Precision irrigation is an area of automation that has received a lot of attention. Dwindling water supplies, prolonged droughts and climatic uncertainties are pushing growers to improve their water use efficiency. In this context, precision irrigation defined as site-specific irrigation management that relies on the variable application of water, emerges as a potential solution to increase the productivity, and reduce the environmental impact, of irrigated agriculture (Monaghan et al., 2012). Despite widespread international use of the term PI, the concept is still in its infancy and its adoption outside research is very limited (Daccache et al., 2015). Wireless sensor networks have been developed to sense water-stress related variables and actuate valves (Coates et al., 2013; Dhillon et al., 2017). One of the main factors affecting the adoption of precision irrigation is the lack of a flexible, reliable and intelligent variable rate irrigation (VRI) system that can respond to the spatial field variability. Therefore, monitoring (moisture and water stress sensors) and controlling technologies (i.e. individually controlled sprinklers and/or variable speed pulls) combined with decision support tools will be needed to ensure an appropriate operation of the system and to quantify the potential economic and environmental benefits of such system (Daccache et al., 2015).
Using qualitative interviewing and analysis, Caplan et al. (2014) used a diffusion of innovations framework to gain insight into what channels of communications impacted planned adoption rates and what aspects of technology influence the decision-making process. Interviews of participants emphasized the inevitability of implementing new technologies while considering the capital investment of more complex technology, changes in labor management to integrate technology, applicability of technology to current practices, and trust in technology designers.
A systems approach is needed to solve specialty crop labor intensive and crop sensing challenges. A key theme in this project is investigating the whole system, ranging from identifying key biological parameters of specialty crops through the commercialization of new products. Our long-term goals are increased production efficiency, profitability, environmental stewardship, and social responsibility in specialty crop systems.
Adapt biological concepts associated with specialty crop production, harvest, and postharvest handling into quantifiable parameters which can be sensed
Develop specialty crop architectures and systems that are more amenable to mechanized production
Study interactions between machinery and crop to provide basis for creating optimal mechanical and/or automated solutions for specialty crop production
Develop sensors and sensing systems which can phenotype and measure and interpret quality parameters
Design and evaluate automation systems which incorporate varying degrees of mechanization and sensors to assist specialty crop industries with labor, management decisions, and reduction of production costs
Develop collaboration and work in partnership with equipment and technology manufacturers to commercialize and implement the outcomes of this project
Multiple Crop and Cross Platform Integration: A key and continuing theme in this project is a multi dimensional integration of research activities; integration from the adaptation of biological concepts to measurable parameters through the commercialization of new products; and integration within each objective among different specialty crops. Although the objectives seem sequential in execution, we anticipate a swirl of concurrent activities centered about and driven by the needs of specialty crop growers for automation of growing, harvesting and postharvest operations. A major task for the project leadership is to facilitate communication and collaboration among the members of this project, and between project members and other stakeholders in specialty crop agriculture.
Objective 1: Adapt biological concepts into parameters that can be sensed.
Every automation system for specialty crops will depend on sensing both the immediate environment and the "biological state" (the combination of genetics, growth history including pesticide pressures, and current environmental conditions which influence future growth and quality factors) of the plant and produce. Developing the knowledge relating the biological state to parameters that can be externally measured is central to sensor design.
Relating the biological state into measurable parameters is a direct continuation of the research conducted under NE-179 and NE-1008, the predecessors of the proposed project, and there is a deep level of expertise among the members of this proposed project in the areas of physical properties, enzymatic reactions, internal quality and plant response to insect or disease attack and environmental stress.
Defining how external loads are transferred to cells, and under what conditions will those loads result in mechanical failure and/or injury response from the plant has been studied in almost every experiment station represented in this project. Continuing and future research will define the cellular mechanical properties, and how these cell-scale properties integrate to form tissue-scale mechanical properties that can be linked to quality parameters such as firmness and crispness.
Optical and electrical properties will be applied to assess internal and external quality factors, such as the amount of soluble sugar, and the maturity level of many fruits and vegetables. Optical, NMR, X-ray, infrared and other electromagnetic techniques will be applied to determine the relationship between internal quality factors and consumer acceptance of various produce using, with focus on development of rapid, cost effective, and more accurate methods and techniques for quantification of the electromagnetic properties and their relationship with internal quality parameters of fruits and vegetables.
The biological concepts described above do not act in isolation to each other. Causal relationships such as enzyme activity and cell mechanical properties exist. Abnormal "biological states", such as water deficient stress, often cause multiple responses among the concepts described above. Investigations will address using combinations of these concepts to gain more precise and insightful understanding of the "biological state". For example, research into plant response to insect/disease attack and environmental stress will provide a rapid and strong indicator that intervention is required.
Objective 2: Develop new crop production architectures and systems
For mechanization and automation to be successful, the crop architecture must enable machinery to easily access the plants. Conventional orchard layouts with large, wide (i.e. three-dimensional) trees make mechanized tasks very difficult. Research into novel architectures will continue, with focus on changing the structures to a more two-dimensional configuration, which makes many production tasks simpler (such as thinning, pruning, scouting, and harvest). The orchard architecture developments will be addressed through commercially viable approaches. New and innovative technologies have been proven more successful with the change in structures. An example is mechanical blossom thinning using a string thinner. Proper training and pruning helped to optimize the thinning levels, resulting in reduced labor and higher quality yields. Crop architectures will continue to be investigated and training systems will be further refined to achieve high productivity, low canopy training/management costs and improved interaction with machinery. This objective focuses on the integration of biology, horticulture, and production economics. The study of machine-canopy interaction constitutes an objective on its own (next).
Objective 3: Study interactions between machinery and crops
Participants: Heinemann (PA), all
Many of the interactions between the crop and machinery (such as pruning, thinning, harvesting, sorting, storage, and transportation) will involve physical contact. Most of these interactions must be non destructive to the plant. Sensing of mechanical properties to establish maximum forces that may be applied without damage will be investigated and determined. Research and design is required to develop sensors that can nondestructively, rapidly and accurately determine the inherent strength of plant tissue.
Mechanical or automated harvesting of fruit crops is a critical issue in specialty crop production. Experiments in mechanical impact and vibration energy transmission through canopies in different types of crop architectures will be carried out. This will provide valuable information for developing efficient and effective interfaces for existing mechanical harvesters and new technologies for mechanical and automated harvesting. Accessibility of machines to fruits, flowers and branches in different types fruit trees and bushes will be studied, which will be beneficial for improving or developing new technologies for not only harvesting, but also for pruning and thinning. Chemical application by fixed and mobile spraying systems that move in various types of crop canopies will be investigated to improve technologies for better coverage and reduced drift and off-site movement. Multi-disciplinary work to simultaneously improve machines and crop architectures will be a key for the success in these areas. In addition to ‘traditional’ methods based on field-testing, model-based approaches will also be pursued. Tree canopies will be digitized using high-resolution scanners and canopy-fruit-machine interactions (e.g., ease of canopy penetration, fruit reachability) will be investigated using software and computational geometry methods.
Objective 4: Develop sensors and sensing systems which can phenotype and measure and interpret quality parameters
Participants: Heinemann (PA), all
There is a long and rich history of sensor research and development aiming toward sensing the "biological state". New applications of established sensor types and novel sensors will be applied for phenotyping and also for assessing quality parameters. Most of the sensors developed to date are used in more typical controlled environments, so these sensors must be rugged and adaptable for field and packinghouse use. The multitude of sensor designs can be roughly classified by three human senses; touch, sight and smell.
Image and vision-based sensors will provide the bulk of the information to an automation system. Vision systems (including cameras, NMR devices, IR devices and other devices based on radiated energy) will be focused on a variety of targets, work non-invasively, provide information in the visible and non-visible spectrums, and provide a large amount of data. The challenge in using vision systems is to extract the required information from the collected data. For some imaging devices such NMR and X-ray, moving to a cost effective, portable system in an outdoor environment will be a challenge in itself. New concepts such as stereoscopic images, coupled with an ever-increasing amount of portable computing power will be applied to extract the information, but there is a significant challenge in designing effective vision-based sensing systems at an acceptable cost. New tools from deep learning neural networks will be also utilized to address this challenge.
Objective 5: Design and evaluate automation systems
Participants: Heinemann (PA), all
A great deal of knowledge and equipment from the industrial and military use of automated equipment can be applied to specialty crops, but the wide variation and semi-chaotic nature of field operations provide a significant number of research challenges. A key challenge is adaptability to multiple crops and cropping systems. Applications of new automation technologies and autonomous controls will be investigated to improve the efficacy of automated systems. These will be developed with cost constraints in mind, as growers can not necessarily afford high cost solutions. The capital investment of automated equipment is high, and the potential market is small. It is critical to spread research and development costs by developing automated systems that can easily adapt to different crops and different cropping systems.
An automated system for specialty crops is much more than an autonomous robot moving along the row of trees or plants. A cost effective system will require information from many sources both on-site and off-site, and autonomous robots or vehicles will require direction and coordination. Hence the efficient gathering of data and control of devices will be an integral part of an automated system. New autonomous orchard vehicles have been extensively tested and proven in the field. Further work can investigate integration of tasks with these vehicles and platforms, as well as fully automating certain operations, which have not yet been successfully addressed, such as autonomous fresh fruit harvesting.
It is not feasible to replace human labor with automation in all operations. In some operations the cost of replacing human dexterity and complex decision-making capability with equipment is not justified. In these operations, a semi-automated device assisting human labor may be a better, more cost-effective solution. Developing a man-machine system requires careful attention to the ergonomic (safety, productivity, comfort and intellectual engagement) needs of the human in the system. Very little research has been done in ergonomic design of specialty crop equipment, and additional research is required to design an optimum man-machine system for specialty crops. Furthermore, although there has been extensive research and development of farm robotics, very little specifically addresses the safety of farmworkers working with robots. Thus, research must be undertaken to establish safety and health standards and best practices in both incorporating robots into farms. Such research needs to do the following: monitor trends in worker injuries; identify risk factors for farmworkers’ injuries when working with robots; research interactions between humans and robots and optimal designs for the human-robot interface, and develop and evaluate interventions to prevent farmworkers’ injuries.
Decisions regarding the overall plan and control of an automated system will remain with a human operator. The quantity of operator decision-making information needed for autonomous or semi-autonomous operation of various equipment may potentially be unprecedented and unmanageable. There is a need to develop computer-based data systems which collect, verify, and organize raw data to present information to the operator such as the maturity of the crop, crop stress, and spatial variation. There is also a need to organize predictive models for crop needs such as pesticide application, pruning, thinning and harvesting. Finally, there is also a need to help the operator visualize the most effective use of the automated equipment.
Objective 6: Develop collaborations and partner with equipment and technology manufacturers in commercializing
The selection of a particular crop and operation covered by this project will be based on growers' needs for automation of that crop and operation. It is important to take an integrated approach and for researchers working on different aspects of specialty crop production to come together and collaborate to develop real-world solutions. The areas of research may include horticulture, agricultural and biological engineering, plant and soil science, plant pathology, food quality and safety, postharvest biology and technologies, water resources, agricultural economics, environmental science, and mechanical engineering, and not limited to those mentioned above. With that as a premise, we cannot consider the research complete until a piece of equipment or device is commercialized and in use. Immediately following acceptance of this project, we will initiate meetings between project participants, specialty crop growers and manufacturers to target specific crops and operations for which we will develop automation.
Commercialization of many critical automation needs of specialty crop growers will be difficult to justify using traditional business plans because the small size of the market, coupled with potentially high development costs, will result in a low return on investment (ROI). Manufacturers will need help to lessen the cost of development and risk assumed to marketing a new product. Frequent communication between growers, manufacturers and project participants will help to mitigate cost and risk to the manufacturers. We will create opportunities for our partnerships with specialty crop growers and manufacturers (currently, companies like Deere & Company, Oxbo International Corp., Durand-Wayland, Inc., Trimble Navigation Ltd., DBR Conveyance Concepts, COE Orchard Eq., Orchard machinery Corporation, Atlas Pacific Engineering and others have partnered with project participants in various projects and/or programs).
Defining a set of industry standards for the test, operation and components of automated equipment is a necessity. Standards will enable the interchangeability of parts and software, decrease design time, and encourage manufacturers to build component parts such as articulating arms and end effectors. We will work with the standards committees of ASABE and SAE to propose standards related to automation in specialty crops.
The design of automation systems is a continually evolving engineering area, and the application of automation design to specialty crops is very new. Scientific findings along with engineering concepts and techniques learned during this project should be shared among practicing scientists, engineers and students. During the project we will periodically collect concepts and techniques learned from among the participants, and disseminate this knowledge through classroom and continuing education venues.
Measurement of Progress and Results
- Production structures and systems that fit mechanization and automation
- Sensors capable of measuring the "biological state" adapted for outdoor use on automated equipment
- Sensors used in industrial and military automation adapted for use in specialty crop environments
- Sensors capable of measuring and monitoring product quality and food safety during harvest and postharvest operations.
- Specialty crop automated and semi-automated equipment available
- Wide-area specialty crop data communication systems available
- Decision-making software for use in aiding the management of automated equipment
- Engineering models relating mechanical and/or physical properties, enzymatic reactions, internal quality and plant external stress indicators to the "biological state" of selected specialty crops
- Engineering models which estimate "biological state" based on the interaction of multiple indicators
- Integrated set of design, test and manufacturing standards
- Design, manufacturing and usage education modules for use in university and continuing education learning.
Outcomes or Projected Impacts
- Specialty crop technology development
- Modernized, mechanization compatible crop production designs
- Research publications in the design of specialty crop technologies
- Training of graduate and undergraduate students in the design and concepts of specialty crop automated equipment
- Workshops and other continuing education opportunities for practicing scientists and engineers
- Competitive advantage for domestic specialty crop producers by increasing labor efficiency with automated equipment and systems
- Healthier and safer working environment for the remaining human workforce used in specialty crop production and handling
- Manufacturing workforce (design engineers, mechanics, operators) better prepared to manufacture and use automated equipment for specialty crop production, postharvest storage, processing, handling, and sorting
- Reduction of the impact of specialty crop production on the environment through more precise field, postharvest facilities, and packinghouse operations
Milestones(2018):(2018): Spring: Meetings between researchers, growers and manufactures have identified targeted specialty crops and related operations. Summer 2018: Research, design and manufacturing responsibilities for the targeted crops / operations assigned. Fall 2018: Target crops / operations requiring longer development time (not likely to be completed within the 5 year duration of this project) identified. Year 1: Identify emerging technologies in specialty crop and forecast future research needs
(2019):(2019): Spring 2019: Obtaining industrial and federal grants to support team research. Year 2: Organize systems-driven research and extension teams on each focus area and develop technology elements
(2020):(2020): Spring 2020: Developing prototypes and conducting field research. Year 3: Create testbeds to transfer new technologies developed by the teams
(2021):(2021): Spring 2021: Filing patent application for developed technologies. Year 4: Develop strong commercialization plans and support industry to adopt developed technologies
(2022):(2022): Spring 2022: Organizing workshops and helping industry to commercialize and apply the technologies. Fall 2022: Project ends (renewal in place)
Projected ParticipationView Appendix E: Participation
From the user-centered design nature of this project, outreach to our partners is continuous and integral to the project. The intended users will be a part of the initial meetings to select targeted crops / operations, in the prototype design and testing of equipment, in the development of business plan and marketing tools, and in the commercialization of the target equipment. Indicators of the project outreach will include the expected scientific papers, patents and publications in the trade and popular press. In addition, there will be frequent communication with manufactures, growers and the people who will be using the automated equipment, sets of standards to convey the design concepts learned to a wide audience of engineers and technicians, and educational modules which can be used in classroom and continuing education venues.
The technical committee will consist of project leaders for the contributing states, the administrative advisor, and CSREES representatives. Voting membership includes all persons with contributing projects.
A Chairperson, Vice Chairperson and Secretary will be elected from the voting membership at the first authorized committee meeting after the project has been approved. The Chair, Vice-Chair and Secretary will serve two years, if so desired by the membership. They will be responsible for meeting arrangements, annual reports, implementation of the Outreach Plan, and preparation of the renewal proposal.
Due to the number and diversity of the membership, and the user-based nature of this project, a working group and coordinator for each target crop / operation will be selected by the Chairperson from among the membership. The coordinator will be responsible for communication within the working group, developing a timeline for the targeted crop / operation, and coordinating activities among the four project objectives (as apply to the particular targeted crop / operation).
Arikapudi, R., Vougioukas, S.G., Jiménez- Jiménez, F., Farangis Khosro Anjom, F. (2016). Estimation of Fruit Locations in Orchard Tree Canopies Using Radio Signal Ranging and Trilateration. Computers and Electronics in Agriculture (125):160-172. http://dx.doi.org/10.1016/j.compag.2016.05.004
Bao, Y., L. Tang. 2016. Field-based Robotic Phenotyping for Sorghum Biomass Yield Component Traits Characterization Using Stereo Vision. IFAC-PapersOnLine. 49(16):265-270. https://doi.org/10.1016/j.ifacol.2016.10.049
Baugher, T. Auxt. 2003. Training Systems. In: Baugher, T. Auxt and S. Singha (editors). 2003. Concise Encyclopedia of Temperate Tree Fruit. Haworth Press, Inc., NY. 387 pp.
Bergerman, M., S. Singh, and B. Hamner. 2012. Results with autonomous vehicles operating in specialty crops. IEEE International Conference on Robotics and Automation. Saint Paul, Minnesota. May 14-15. Pp. 1829-1835.
Bloch, V., Degani, A., Bechar, A. 2018. A methodology of orchard architecture design for an optimal harvesting robot, Biosystems Engineering (166):126-137.
Caplan, S., B. Tilt, G. Hoheisel, T. Auxt Baugher. 2014. Specialty crop growers’ perspectives on adopting new technologies. HortTechnology 24:81-87.
Castillo-Ruiz, F. J., Sola-Guirado, R. R., Castro-Garcia, S., Gonzalez-Sanchez, E. J., Colmenero-Martinez, J. T. & Blanco-Roldan, G. L. 2017. Pruning systems to adapt traditional olive orchards to new integral harvesters. Scientia Horticulturae, 220: 122-129.
Chen, S. W., Shivakumar, S. S., Dcunha, S., Das, J., Okon, E., Qu, C., Taylor, C. J. & Kumar, V. 2017. Counting Apples and Oranges with Deep Learning: A Data-Driven Approach. Ieee Robotics and Automation Letters, 2(2): 781-788.
Choi, D., Lee, W. S., Schueller, J. K., Ehsani, R., Roka, F., & Diamond, J. 2017. A performance comparison of RGB, NIR, and depth images in immature citrus detection using deep learning algorithms for yield prediction. 2017 ASABE Annual International Meeting, Paper No. 1700076, pages 1-6, July 16 – 19, 2017.
Choi, D., Lee, W. S., Schueller, J. K., Ehsani, R., Roka, F., & Ritenour, M. 2016. A precise fruit inspection system for Huanglongbing and other common citrus defects using GPU and deep learning technologies. 13th International Conference on Precision Agriculture, Paper No. 1835, pages 1-6, July 31 - August 4, 2016.
Coates, R. W., Delwiche, M. J., Broad, A., & Holler, M. (2013). Wireless sensor network with irrigation valve control. Computers and electronics in agriculture, 96, 13-22.
Constante, P., Gordon, A., Chang, O., Pruna, E., Escobar, I. & Acuna, F. 2016. Artificial Vision Techniques for Strawberry's Industrial Classification. IEEE Latin America Transactions, 14(6): 2576-2581.
Cortes, V., Blanes, C., Blasco, J., Ortiz, C., Aleixos, N., Mellado, M., Cubero, S. & Talens, P. 2017. Integration of simultaneous tactile sensing and visible and near-infrared reflectance spectroscopy in a robot gripper for mango quality assessment. Biosystems Engineering, 162: 112-123.
Crassweller, R.M. and D.E. Smith. 2013. Second generation apple systems trials. PA Fruit News 93 (1):67-71.
Cubero, S., Aleixos, N., Moltó, E., Gómez-Sanchis, J., Blasco, J., 2010. Advances in Machine Vision Applications for Automatic Inspection and Quality Evaluation of Fruits and Vegetables. Food and Bioprocess Technology 4:487-504.
Daccache, A, Knox, J.W., Weatherhead, E.K., Daneshkhah, A., and Hess, T. (2015). Implementing precision irrigation in a humid climate - recent experiences and on-going challenges. Agricultural Water management. Volume 147 Pages 135–143
Dhillon, R. S., Upadhaya, S. K., Rojo, F., Roach, J., Coates, R. W., & Delwiche, M. J. (2017). Development of a Continuous Leaf Monitoring System to Predict Plant Water Status. Transactions of the ASABE, 60(5), 1445-1455.
Donis-González, I.R., Guyer, D.E., Pease, A., 2012a. Application of Response Surface Methodology to systematically optimize image quality in computer tomography: A case study using fresh chestnuts (Castanea spp.). Computers and Electronics in Agriculture 87:94-107.
Donis-González, I.R., Guyer, D.E., Pease, A., Fulbright, D.W., 2012b. Relation of computerized tomography Hounsfield unit measurements and internal components of fresh chestnuts (Castanea spp.). Postharvest Biology and Technology 64:74-82.
Donis-González, I.R., Guyer, D.E., Fulbright, D.W., Pease, A., 2014a. Postharvest noninvasive assessment of fresh chestnut (Castanea spp.) internal decay using computer tomography images. Postharvest Biology and Technology 94: 14-25.
Donis-González, I.R., Guyer, D.E., Chen, R., Pease, A., 2015a. Evaluation of undesirable fibrous tissue in processing carrots using Computed Tomography (CT) and structural fiber biochemistry. Journal of Food Engineering 153:108-116.
Dvorak, J., Stone, M., & Self, K. (2016) Object Detection for Agricultural and Construction Environments using an Ultrasonic Sensor. Journal of Agricultural Safety and Health. 22(2): 107-119. doi: 10.13031/jash.22.11260.
Durand-Petiteville, A., Vougioukas, S. G., Slaughter, D.C. (2017). Real-time segmentation of strawberry flesh and calyx from images of singulated strawberries during postharvest processing. Computers and Electronics in Agriculture, (142A): 298-313. https://doi.org/10.1016/j.compag.2017.09.011
Espinoza, C. Z., Khot, L. R., Sankaran, S. & Jacoby, P. W. 2017. High Resolution Multispectral and Thermal Remote Sensing-Based Water Stress Assessment in Subsurface Irrigated Grapevines. Remote Sensing, 9(9): 961, pp 1-15.
Everard, C., M. Kim, M.C. Siemens, H. Cho, H., A. LeCourt, and C. O’Donnell. 2017. A multispectral imaging system using solar illumination to distinguish fecal matter on leafy greens and soils. Biosystems Eng. (submitted)
Fan, S. X., Li, C. Y., Huang, W. Q. & Chen, L. P. 2017. Detection of blueberry internal bruising over time using NIR hyperspectral reflectance imaging with optimum wavelengths. Postharvest Biology and Technology, 134:55-66.
Fennimore, S.A., B. D. Hanson, L. M. Sosnoskie, J. B. Samtani, A. Datta, S. Z. Knezevic, and M. C. Siemens. 2013. Chapter 9: Field Applications of Automated Weed Control: Western Hemisphere. In Automation: The Future of Weed Control in Cropping Systems, 151-169. S.L. Young and F.J. Pierce, eds. Dordrecht: Springer Science+Business Media.
Fennimore, S.F., D.C. Slaughter, M.C. Siemens, R.G. Leon, and M.N. Saber. 2016. Technology for Automation of Weed Control in Specialty Crops. Weed Technology, 30(4): 823-837.
Gan, H., Lee, W. S., & Alchanatis, V. (2017). A Prototype of an Immature Citrus Fruit Yield Mapping System. In 2017 ASABE Annual International Meeting (p. 1). American Society of Agricultural and Biological Engineers.
Gongal, A., Silwal, A., Amatya, S., Karkee, M., Zhang, Q., & Lewis, K. (2016). Apple crop-load estimation with over-the-row machine vision system. Computers and Electronics in Agriculture, 120, 26-35.
He, L., Fu, H., Sun, D., Karkee, M. & Zhang, Q. 2017. Shake-and-Catch Harvesting for Fresh Market Apples in Trellis-Trained Trees. Transactions of the ASABE, 60(2): 353-360.
Hung, C., Underwood, J., Nieto, J., & Sukkarieh, S. (2015). A feature learning based approach for automated fruit yield estimation. In Field and Service Robotics (pp. 485-498). Springer International Publishing.
Jackson, J. & Dvorak, J. (2016). Hybrid Diesel-Electric Drivetrain for Small Agricultural Field Machines. Transactions of the ASABE. 59 (5). 1117-1125. doi: 10.13031/trans.59.11623
Khosro Anjom, F., Vougioukas, S. G., Slaughter, D.C. (2018). Development of a Linear Mixed Model to Predict the Picking Time in Strawberry Harvesting Processes. Biosystems Engineering. (166): 76-89. https://doi.org/10.1016/j.biosystemseng.2017.10.006
Kon, T. M., J. R. Schupp, H. E. Winzeler and R. P. Marini. 2013. Influence of mechanical string thinning treatments on vegetative and reproductive tissues, fruit set, yield and fruit quality of apple. HortScience 48:40-46.
Larbi, P. A., Karkee, M., Amatya, S., Zhang, Q. & Whiting, M. D. 2015. Modification and field evaluation of an experimental mechanical sweet cherry harvester. Applied Engineering in Agriculture, 31(3): 387-397.
Lefcourt, A.M. and M.C. Siemens. 2017. Interactions of insolation and shading on ability to use fluorescence imaging to detect fecal contaminated spinach. Appl. Sci., 7, 1041.
Li and Tang (2017) developed a crop plant recognition algorithm to identify green bean and broccoli plants in weedy conditions by using 3D images.
Li, J. B., Chen, L. P. & Huang, W. Q. 2018. Detection of early bruises on peaches (Amygdalus persica L.) using hyperspectral imaging coupled with improved watershed segmentation algorithm. Postharvest Biology and Technology, 135: 104-113.
Li, J., L. Tang. 2017. Crop recognition under weedy conditions based on 3D imaging for robotic weed control. Journal of Field Robotics. 2017:1-16
Li, J., L. Tang. 2017. Developing a low-cost 3D plant morphological traits characterization system. Computers and Electronics in Agriculture 143:1-13.
Lu, H., L. Tang, S.A. Whitham, Y. Mei. 2017. A Robotic Platform for Corn Seedling Morphological Traits Characterization. Sensors. 17(9), 2082.
Lyons, D.J., P.H. Heinemann, J. Liu, J., J.R. Schupp, and T.A. Baugher. 2015. Development of a selective automated blossom thinning system for peaches. Transactions of ASABE. 58(6):1447-1457.)
Miller, S., J. Schupp, T. Baugher, and S. Wolford. 2011. Performance of mechanical thinners for bloom or green fruit thinning in peaches. Hort-Science 46:43–51.
Monaghan J., Daccache A., Vickers L., Hess T., Grove Iand Knox J. (2012). More ‘crop per drop’ – constraints and opportunities for precision irrigation in European agriculture. Journal of the science of food and agriculture. Volume 93, Issue 5: 977–980.
Munera, S., Amigo, J. M., Blasco, J., Cubero, S., Talens, P. & Aleixos, N. 2017. Ripeness monitoring of two cultivars of nectarine using VIS-NIR hyperspectral reflectance imaging. Journal of Food Engineering, 214: 29-39.
Nguyen, T. T., Slaughter, D. C., Max, N., Maloof, J. N., & Sinha, N. (2015). Structured light-based 3D reconstruction system for plants. Sensors, 15(8), 18587-18612.
Nguyen, T. T., Slaughter, D. C., Townsley, B., Carriedo, L., Julin, N. N., & Sinha, N. (2016). Comparison of Structure-from-Motion and Stereo Vision Techniques for Full In-Field 3D Reconstruction and Phenotyping of Plants: An Investigation in Sunflower. In 2016 ASABE Annual International Meeting (p. 1).
Oberti, R., Marchi, M., Tirelli, P., Calcante, A., Iriti, M., Tona, E., Hocevar, M., Baur, J., Pfaff, J., Schutz, C. & Ulbrich, H. 2016. Selective spraying of grapevines for disease control using a modular agricultural robot. Biosystems Engineering, 146: 203-215.
Ran N. Lati, M.C. Siemens, J.S. Rachuy, and S.A. Fennimore. (2016). Intrarow Weed Removal in Broccoli and Transplanted Lettuce with an Intelligent Cultivator. Weed Technology, 30(3): 655-663.
Romero-Trigueros, C., Nortes, P. A., Alarcon, J. J., Hunink, J. E., Parra, M., Contreras, S., Droogers, P. & Nicolas, E. 2017. Effects of saline reclaimed waters and deficit irrigation on Citrus physiology assessed by UAV remote sensing. Agricultural Water Management, 183: 60-69.
Rounsaville, J., Dvorak, J., Stombaugh, T. (2016). Methods for Calculating Relative Cross-Track Error for ASABE/ISO 12188-2 from Discrete Measurements. Transactions of the ASABE. 59(6): 1609-1616. doi: 10.13031/trans.59.11902
Sa, I., Ge, Z. Y., Dayoub, F., Upcroft, B., Perez, T. & McCool, C. 2016. DeepFruits: A Fruit Detection System Using Deep Neural Networks. Sensors, 16(8): 1222, pp. 1-23.
Schupp, J., and T. Baugher. 2011. Peach blossom string thinner performance improved with selective pruning. HortScience. 46:1486-1492.
Schupp, J., H. E. Winzeler, T. M. Kon, R. P. Marini, T. A. Baugher, L. F. Kime, and M. A. Schupp. 2017. A method for quantifying whole-tree pruning severity in mature tall spindle apple plantings. HortScience 52:1233-1240.
Senthilnath, J., Dokania, A., Kandukuri, M., Ramesh, K. N., Anand, G. & Omkar, S. N. 2016. Detection of tomatoes using spectral-spatial methods in remotely sensed RGB images captured by UAV. Biosystems Engineering, 146: 16-32.
[endif]Seyyedhasani, H., Dvorak, J., Sama, M., & Stombaugh, T. (2016). Technical Note: Mobile device-based location services accuracy. Applied Engineering in Agriculture. 32(5). 539-547. doi: 10.13031/aea.32.11351
Seyyedhasani, H., Dvorak, J. (2017). Using the Vehicle Routing Problem to Reduce Field Completion Times with Multiple Machines. Computers and Electronics in Agriculture. 134. March 2017. 142-150. http://dx.doi.org/10.1016/j.compag.2016.11.010
Shah, D., L. Tang, J. Gai, R. Putta-Venkata. 2016. Development of a mobile robotic phenotyping system for growth chamber-based studies of genotype x environment interactions. IFAC-PapersOnLine. 49(16):248-253. https://doi.org/10.1016/j.ifacol.2016.10.046
Siemens, M.C. Robotic weed control. 2014. In Proc. 66th Annual California Weed Science Society 66: 76-80. Salinas, Calif.: California Weed Science Society.
Siemens, M.C., and R.R. Gayler, R.R. 2016. Improving seed spacing uniformity of precision vegetable planters. Appl. Eng. Agric., 32(5): 579-587.
Siemens, M.C. & Gayler. R.R. (2016). Alternative systems for cultivating and side dressing specialty crops for improved nitrogen use efficiency. ASABE paper No. 162456725, pp. 9. St. Joseph, Mich.: ASABE.
Silwal, A., Davidson, J. R., Karkee, M., Mo, C. K., Zhang, Q. & Lewis, K. 2017. Design, integration, and field evaluation of a robotic apple harvester. Journal of Field Robotics, 34(6): 1140-1159.
Tabb, A. and K. M. Ahmad Yousef, "Parameterizations for reducing camera reprojection error for robot-world hand-eye calibration," 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Hamburg, 2015, pp. 3030-3037. doi: 10.1109/IROS.2015.7353795
Tabb, A. (2017). Data from: Solving the Robot-World Hand-Eye(s) Calibration Problem with Iterative Methods. Ag Data Commons. http://dx.doi.org/10.15482/USDA.ADC/1340592
Tabb, A. 2013. Shape from Silhouette Probability Maps: Reconstruction of Thin Objects in the Presence of Silhouette Extraction and Calibration Error. In Proceedings of the 2013 IEEE Conference on Computer Vision and Pattern Recognition (CVPR '13). IEEE Computer Society, Washington, DC, USA, 161-168. DOI: http://dx.doi.org/10.1109/CVPR.2013.28
Tabb, A. and H. Medeiros, "A robotic vision system to measure tree traits," 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Vancouver, 2017.
Tabb, A. and J. Park, "Camera calibration correction in Shape from Inconsistent Silhouette," 2015 IEEE International Conference on Robotics and Automation (ICRA), Seattle, WA, 2015, pp. 4827-4834. doi: 10.1109/ICRA.2015.7139870
Tabb, A. and K. M. Ahmad Yousef. 2017. Solving the robot-world hand-eye(s) calibration problem with iterative methods. Mach. Vision Appl. 28, 5-6 (August 2017), 569-590. DOI: https://doi.org/10.1007/s00138-017-0841-7
Tang et al. (2015) developed a real-time crop stand analyzer for high-throughput infield crop sensing.
Tang, L., J. Li, Y. Bao, J. Jin, A.D. Nakarmi. 2017. Crop stand analyzer using reflective laser proximity sensors. US Patent 9,804,097
Vougioukas, S., Spanomitros, Y., Slaughter, D. (2012). Dispatching and Routing of Robotic Crop-transport Aids for Fruit Pickers using Mixed Integer Programming. ASABE Annual International Meeting Paper Number 21338247, Dallas, Texas.
Vougioukas, S.G., Arikapudi R., Munic, J. (2016). A Study of Fruit Reachability in Orchard Trees by Linear-Only Motion. Journal: IFAC-PapersOnLine, 49(16), pp.277-280. (From Peer reviewed proceedings of the 5th Agricontrol Conference, Seattle, USA.) https://doi.org/10.1016/j.ifacol.2016.10.051
Ye, Y., He, L., & Zhang, Q. (2016). Steering Control Strategies for a Four-Wheel-Independent-Steering Bin Managing Robot. IFAC-PapersOnLine, 49(16), 39-44.
Ye, Y., Wang, Z., Jones, D., He, L., Taylor, M. E., Hollinger, G. A., & Zhang, Q. (2017). Bin-Dog: A Robotic Platform for Bin Management in Orchards. Robotics, 6(2), 12.
Zhang, Z., P.H. Heinemann, J. Liu, J.R. Schupp, and T.A. Baugher. 2016. Design and field test of a low-cost apple harvest-assist unit. Transactions of ASABE. 59(5):1165-1180