Digital Observer Project - Software Development. PI: Dr. Mitchell Roth, Department of Mathematical Sciences, UAF Budget. $50,368.43 Abstract: This software development project is a required and integral part of a larger independently funded project called the Digital Observer. In essence, the Digital Observer consists of an automated video camera system poised above the deck of a fish harvesting vessel. At short intervals images are taken and captured. The images will be electronically impressed with time, date, and GPS derived vessel position information. These images will be processed by software developed in this project. The new software will modify existing computer code to allow it to identify to species and size fish as they come aboard from longline gear. Regulations require that a significant proportion of harvesting vessels be manned by observers. Observers record harvests taken by the vessel including target species and bycatch. One major dilemma in fisheries in Alaska is that regulations specify that all harvesting in federal waters must stop when the bycatch of certain species such as halibut exceed specific levels. Another problem is that it is increasingly difficult to find qualified observers for this work, hindering management. The final problem is that keeping an observer on board a vessel is costly and vessel owners and skippers would like to reduce their costs. The Digital Observer Project, if successful, would solve these problems. Background The Digital Observer Project seeks to reduce the need for - and in some cases replace - onboard observers in the world's longline fisheries by employing existing technology that is enhanced by specially-developed software. The organization's immediate goal is to demonstrate that machines can identify longline-caught fish as they are hauled from the water. Its ultimate goal is deploy "smart" machines on boats. These machines will be designed to perform observer functions at a fraction of the current cost, without as much burden on the fishermen and with high accuracy. In 1998 observer costs aboard the 186 Alaska longliners that carried observers exceeded $2 million and as a whole, government and industry around the world pay $10-12 million per year for observer coverage aboard longliners. When the system is adopted, benefits to payers will be dramatically lowered observer costs coupled with access to accurate, real time production statistics. The system will also ease the industry-wide observer labor shortage, freeing up observers for boats where the system is not yet in use. Government and those with environmental concerns will benefit through assured regulatory compliance and an improved data stream. The world's longline fisheries are coming under increasing scrutiny lately, due to the bycatch of such creatures as seabirds, sea turtles, and marine mammals. One way to assess fishing's impact on non-target species is to put human observers on fishing boats. The observers will assure the boat fishes in compliance with the regulations and will also collect biological data. These observers, however, are expensive; costing anywhere from $300 per day in Alaska's fisheries. A rudimentary, prototype system is already operational in B.C's offshore longline sablefish fleet. That system, which is designed to assure regulatory compliance only, uses standard video tapes which document fishing activity. The Canadian system has been in use since 1994 and fishermen like it because it is cheaper and easier to live with than having human observers aboard. British Columbia fishermen report the cost of observer coverage for a 30-day trip dropped from C$10,000 to C$2,000. Officials with Canada's Department of Fisheries and Oceans express enthusiasm for the system and support it. In this project's case, a second set of machines will be added to gather and disseminate biological data. Built to be tamper-proof, that system will consist of a high tech, day/night digital camera connected to "ruggedized" computers that are equipped with special software. That package will include: 1. a fish recognition component, 2. a GPS plotter interface, 3. a security module, 4. a communications package. Fish recognition software has already been developed in a number of places around the world. Among those successfully implemented includes one developed by the Alaska Department of Fish & Game (ADF&G) in cooperation with scientists at the University of Minnesota. This software will allow the computer to recognize species caught while a concurrent program notes location, date and time. On longliners, the camera lens will be mounted on a davit outboard of the hauling station, focusing on each fish as it emerges from the water. Shortly after each hook emerges from the water it will trip a laser signal causing the camera to take a series of pictures. We'll need more than one image of each fish because they will be in motion and will be swinging and flopping on the hook. The images will then go to a computer loaded with a program that will select the best one. That image will then be sent on to another program designed to recognize species. Using telemetry, and National Marine Fisheries Service length-weight data, the computer will estimate each fish's length and weight. A concurrent program will note location, date and time and enter all the information in an electronic logbook. By "observing" 100 percent of the fishing activity, the computer will estimate lengths and weights of every fish caught, even discards and bycatch. It will also count every hook, providing CPUE information. The computer will then summarize the data on spreadsheets, and, through wire or satellite links, report on a periodic basis to authorized persons both near and far. The computer hard drive will retain images it cannot identify for later review by human specialists. Project headquarters will be in Kodiak and most field trials will take place there or in the Gulf of Alaska. Trials on crab boats will take place in the Bering Sea. Plans call for software to be written in Anchorage and Fairbanks, Alaska. Time Frame: We submitted a "proof of concept" grant proposal to the Alaska Science and Technology Foundation (ASTF) on Feb. 27 and are hopeful the ASTF board will decide to fund on April 13. In that is the case, the project hopes to commence in May 2000. Its first acts will be to order hardware and in April or May test that hardware through an experimental fishing permits obtained either from ADF&G or NMFS. In June the project will gather images of fish using the successfully-tested hardware aboard a NMFS sablefish (Anoplopoma fimbria) stock assessment charter. The project will then use those images to determine which software program is best suited to our needs. Field trials using fish caught through an experimental fishing permit provided through NMFS or ADFG will begin in May 2001. The project's report will be available in December 2001. Regulatory Considerations Assuming the system proves effective in the various applications described above, the project will work with state and federal agencies and regulatory bodies to obtain approval for its use in lieu of human observers. We anticipate initial approvals will follow the Canadian model insofar as they will be provisional and experimental. Regarding the system's ultimate application, we intend to offer it as a lower-cost option to using human observers in a number of applications. Looking over the horizon, we plan that over time the system will be approved for widespread use. At this stage our business plan calls for marketing a proprietary software system and customized hardware package. The machines and software will be priced reasonably, to make them a cheaper and less burdensome alternative to having a human onboard. The machines will be leased to the industry through franchisees who may include existing observer companies. We are not seeking to get into the observer business; nor do we seek to put any observer companies out of business. We see the observer companies as allies who will continue to interface with industry. The only difference will be in offering their customers the option of either a machine or a human observer. The project has the support of a broadly diversified coalition comprising industry, academia and government. On March 22, 2000 the Observer Advisory Committee of the North Pacific Fishery Management Council, consisting entirely of industry members, declared its unanimous support for the project. The projects duration is projected at 16.5 months, funding will total $556,255 derived from NMFS experimental fishing permit (value $100,000), National Fish & Wildlife Foundation (contribution of $50,000 and the Alaska Science & Technology Foundation (grant of $355,905). This grant requests an additional $50,368.43 Statement of Work Dr. Mitchell Roth, Professor of Computer Science in the Mathematics Department of the University of Alaska Fairbanks, will advise and work on the project. He will also bring two graduate students to the project who will write software code full time during the summer of 2001 (40 hours per week) and part time (20 hours per week) during the school year at the rate of $15.38 per hour. Dr. Roth and the graduate students, in collaboration with Mr. Patrick Simpson of Scientific Fisheries INC, will develop the fish recognition software code for the project. This proposal will provide one month of Roth's salary and benefits at $10,380.03. It will pay the graduate students nine-month, school year stipends of ~ $12,000 each and three-month summer stipends of ~ $8,000. One additional benefit of the project would be the relatively straight forward additional task of using the Digital Observer information to barcode the more valuable fish as to location of capture, time of capture etc. The barcode coupled with existing technology and a relatively inexpensive chip assembly would allow continual temperature monitoring of each fish helping to insure higher quality. Justification and Need There is across the board support of the concept of the Digital Observer Project from the Alaska Seafood Industry. Federal and State regulators, Seafood Harvesters organizations, Processors, and environmental groups are all in support of the project. The major reason for this depth of support lies in the fact that implementation of the Digital Observer on board harvesting vessels will significantly assist in maintaining sustainable levels of harvest for Alaska's commercial and bycatch species. Harvesters, processors, environmentalists, regulators and managers share the concept of responsible stewardship of Alaska's marine resources. The Digital Observer offers a promising new manner to effect this goal. Deliverables A synopsis of the results of this project will be made available through the University of Alaska Sea Grant College Program communications program. This includes publications, press releases and radio shows and the Alaska Fisheries Development Foundation (AFDF) quarterly newsletter, Lodestar. Furthermore, we plan to report the results through seafood and food service industry trade publications and at technical meetings.