The University of Kentucky Center for Visualization & Virtual Environments

ImageNet 3D Research

A group of army commanders sits in a conference room in Washington D.C. – a world away from a battlefield in Afghanistan. Despite the vast physical distance between soldiers in combat and these army commanders, the commanding officers can oversee military action using unmanned sensors. Although this seems like a movie scene, the ImageNet project at the Vis Center is making this a reality. Researchers from across the University collaborate on ImageNet, with the final goal of improving the Army’s ability to prepare for battle through enhanced data gathering, imaging, and display technologies. The project involves software engineering, networking, computer vision, aerial image gathering through remote-controlled planes, and graphics. ImageNet breaks down barriers between disciplines to create new technology that could never be developed in one area.

Two vehicles will gather data: a LIDAR truck and an unmanned aerial vehicle. For example, a LIDAR sensor scanning truck could drive through a residential neighborhood in Afghanistan. The LIDAR scanner will collect 3D points. The truck is also equipped with a “ladybug” camera that takes 360-degree photos. However, the truck’s camera cannot capture the view from above. Unmanned aerial vehicles (UAVs) will fly over the area taking aerial photographs. A control station will collect the data, apply algorithms to the data to extract meaning, and then pass the meaningful material to a remote control center. The algorithms will automatically extract information, such as any suspicious activity or changes in the area. ImageNet will show the commanders in the remote theater sifted information about the battle zone so that the commander will be informed and able to quickly deploy resources.

With the system gathering so much data, an automatic pipeline needs to reduce manual workload. For instance, a human could never pick through the millions of points in a point cloud. Making sense of the 3D data is Dr. Ruigang Yang’s job; he heads the 3D vision team at UK. Dr. Yang uses the 2D images and 3D point clouds to create 3D semantic models. The team focuses mainly on man-made buildings at the moment. With a semantic model, a computer identifies an object as a house, a wall, a roof, and so on. “That is a classic computer vision problem – how to get a computer to understand what it sees.” Computer scientists have chipped away at this dilemma for years with limited success, but Dr. Yang remains hopeful. Since the 3D vision portion of ImageNet focuses on a specific class of objects, man-made buildings, Dr. Yang believes they will succeed.

ImageNet has many applications, both in war and peacetime. If a commander deploys a sensor to take many photos of an area, the sensor takes so many photos that a commander could not look at each one. Researchers must develop a way to filter the inundation of information so officers focus on useful information. The plan uses computers to develop an automatic pipeline to process semantic information. The computer sorts information by knowing what the points show; for example, the LIDAR sensor creates a 3D point cloud of a house. The computer goes beyond recognizing the points’ locations. Instead, the computer analyzes the point cloud so it understands that the points form a house.

ImageNet is useful for meaningful situation awareness. For example, laser scanning recognizes tiny issues on highways such as small surface cracks or the roadway leaning towards the wrong side. The UK 3D team used the LIDAR truck to scan Nada Tunnel in the Red River Gorge. In the future, they will scan the tunnel again to see if the tunnel’s structure changed. ImageNet monitors a site that is difficult to scan, such as a tunnel, and catches structural changes early.
ImageNet expands the resources decision-makers use. An army commander would know everything of significance happening on a battlefield, the President would see the damage resulting from a natural disaster, and an urban planner would have an accurate model of a proposed development. Researchers at the Vis Center work on a wide variety of projects, but they all share one common goal: seeing the world in a new way. ImageNet will change the way we see our world through its delivery of significant information about a situation.

A New Way To See Product Reviews

The Internet changed the way we shop.  Today we can order anything online, ship it straight to our door, and never set foot in the mall again.  The Internet not only transformed how we make purchases, it also revolutionized how we choose what to buy.  With the Internet, the cost of acquiring information about various products is significantly reduced.  It is easy to go to several different sites and compare prices and reviews of various products.

For example, before you choose a camera to buy, you want to be sure that you are getting your money’s worth so you would read other customers’ reviews.  Some reviews are text only, some have pictures and text, and some are videos.  Since these reviews can become powerful sales tools for companies, Dr. Radhika Santhanam decided to research the effect different types of reviews can have on the consumer’s perception of the product.

To research this, Dr. Santhanam’s graduate students, Pei Xu, Lijuan Wu, and Liang Chen, set up an experimental study of three products.  Each product had a review in three formats – text, text and pictures, and video – though the information used in each format remained the same for each review, regardless of the set up.  They researched, “the extent to which visual media in online product reviews that are given by customers persuade other prospective customers to purchase the product,” Dr. Santhanam explained.

Results show that video reviews are the best tool in online shopping.  There is little difference between the influence of text reviews and that of image-based reviews.  A company’s knowledge of how to best present customer reviews can be an influential sales tool, resulting in improved sales of a product.  Researchers hope to eventually be able to tell a company which review mode will work best to promote sales of a certain product.

Dr. Santhanam hopes to continue research in online product reviews with the University of Kentucky’s Vis Center to improve the visualization used in reviews.  With the development of haptic interfaces, or systems that allow you to touch and manipulate a virtual object, haptic reviews may be even more powerful than video reviews.  As more and more companies close brick and mortar stores and sell exclusively online, customer reviews will drive sales even more.

The Vis Center Welcomes Dr. Ruriko Yoshida

Students in math classes often complain that they will never use their mathematical knowledge outside of school. They may balance a checkbook, but will statistics change the world? Dr. Ruriko Yoshida uses statistics to solve real world problems such as how diseases mutate, how to optimize resources, how to optimize evacuation plans in a case of emergency, and how to develop therapies for special needs children. She studies statistical analysis of genetics, optimization problems, and applications of graphical models.

Dr. Ruriko Yoshida recently joined the faculty of the Vis Center, but has worked in the Statistics Department at the University of Kentucky for the past six years. She began collaborating with Dr. Samson Cheung on a project to optimize the placement of cameras for a security system to use as few cameras as possible, balancing affordability with functionality. Dr. Cheung’s research also involves optimization problems and applying graphical models.

At the Vis Center, Dr. Yoshida will work with Dr. Cheung on his mirror-imaging project. Using a computer image as a mirror image is a useful learning tool for autistic children. The image on the computer “mirror” can be modified to help the child learn via video self-modeling.

Dr. Yoshida applies the same optimization methods and graphical models across disciplines. She is able to use statistics to answer questions in biology, technology, and education. Her research improves the lives of others. She said, “I want to do something good in this society. So I love actually applying some mathematical statistical methods.”

To learn more about Dr. Ruriko Yoshida, press here.

NOTICE: Some websites to which these materials provide links for the convenience of users are not managed by the University of Kentucky. The University does not review, control, or take responsibility for the contents of those sites. Equal Opportunity/Affirmative Action Employer.