Safe at the Finnish Line: Privacy Project Kicks Off Collaboration

He may never command the fame of Linus Torvalds, the father of Linux, but fellow Finn Antti Honkela recently helped clear a big barrier to digital privacy.

The associate professor of data science at the University of Helsinki works on differential privacy, a method for guaranteeing a computation based on personal data will keep that data private. In March, the emerging field made MIT Technology Review’s top 10 list of breakthroughs with the promise of profound impact.

Pieces of the technology are already widely used in smartphones and cloud computing. The 2020 U.S. census will even employ it.

“Differential privacy rests on a strong theoretical foundation, so if you follow the algorithm you get privacy guarantees, but to date the performance cost has been quite significant,” said Honkela.

“Now we could close this gap,” he said of the first project in a broad, multi-year collaboration between NVIDIA and AI researchers in Finland.

100x Speedup for Differential Privacy

Honkela and Niki Loppi, a solutions architect at NVIDIA, demonstrated a way to accelerate training for differential privacy 100x by running it on GPUs.

“We often see these kinds of speedups with GPUs, but the exciting thing here was the penalty for adding differential privacy to standard training was only 2-3x rather than 20x observed on CPU systems,” said Loppi.

Their work shows how to make anonymous versions of highly valuable datasets that currently must remain private because they contain sensitive personal information. Releasing privacy-protected versions of such data would let any AI developer build much better models, accelerating the whole field.

As a follow-up, Loppi’s colleagues at NVIDIA are exploring ways to implement an efficient GPU-accelerated approach for random subsampling in AI training. The work could narrow the performance gap further for implementing enhanced differential privacy.

The effort was the first of many, varied projects in the collaboration between NVIDIA and two powerhouse partners in Finland. The Finnish Center for AI (FCAI) is a national effort that pools top researchers from the University of Helsinki, Aalto University and the VTT Technical Research Center of Finland.

Finland’s national supercomputing center, known as CSC, is the other partner with NVIDIA and FCAI. It will run the group’s research projects on its 2.7-petaflops system that includes 320 NVIDIA V100 Tensor Core GPUs.

A Wide Range of AI Targets

The collaboration in Finland comes on the heels of one forged in January in Modena, Italy. They join a growing global community of NVIDIA AI Technology Centers (NVAITC) driving technology forward.

The Finland work will tap into the many areas of expertise of local partners to drive AI forward. The collaboration between AI researchers and GPU experts “is a good model,” said Honkela, a coordinating professor at FCAI.

“Obviously, researchers have to know the code, but sometimes understanding how to run this work efficiently is a specialty of its own that not all researchers have,” he said.

“Through this cooperation, we are able to boost AI research in Finland and better support local scientists already doing great work in the field,” said Simon See, senior director of NVAITC at NVIDIA.

And who knows what goodness may emerge. Honkela notes that a modern, efficient version of backpropagation, an algorithm at the heart of all neural-network training, was first published in 1970 as a master’s thesis by a University of Helsinki researcher.

The post Safe at the Finnish Line: Privacy Project Kicks Off Collaboration appeared first on The Official NVIDIA Blog.

40 Years on, PAC-MAN Recreated with AI by NVIDIA Researchers

Forty years to the day since PAC-MAN first hit arcades in Japan, and went on to munch a path to global stardom, the retro classic has been reborn, delivered courtesy of AI.

Trained on 50,000 episodes of the game, a powerful new AI model created by NVIDIA Research, called NVIDIA GameGAN, can generate a fully functional version of PAC-MAN — without an underlying game engine. That means that even without understanding a game’s fundamental rules, AI can recreate the game with convincing results.

GameGAN is the first neural network model that mimics a computer game engine by harnessing generative adversarial networks, or GANs. Made up of two competing neural networks, a generator and a discriminator, GAN-based models learn to create new content that’s convincing enough to pass for the original.

“This is the first research to emulate a game engine using GAN-based neural networks,” said Seung-Wook Kim, an NVIDIA researcher and lead author on the project. “We wanted to see whether the AI could learn the rules of an environment just by looking at the screenplay of an agent moving through the game. And it did.”

As an artificial agent plays the GAN-generated game, GameGAN responds to the agent’s actions, generating new frames of the game environment in real time. GameGAN can even generate game layouts it’s never seen before, if trained on screenplays from games with multiple levels or versions.

This capability could be used by game developers to automatically generate layouts for new game levels, as well as by AI researchers to more easily develop simulator systems for training autonomous machines.

“We were blown away when we saw the results, in disbelief that AI could recreate the iconic PAC-MAN experience without a game engine,” said Koichiro Tsutsumi from BANDAI NAMCO Research Inc., the research development company of the game’s publisher BANDAI NAMCO Entertainment Inc., which provided the PAC-MAN data to train GameGAN. “This research presents exciting possibilities to help game developers accelerate the creative process of developing new level layouts, characters and even games.”

We’ll be making our AI tribute to the game available later this year on AI Playground, where anyone can experience our research demos firsthand.

AI Goes Old School

PAC-MAN enthusiasts once had to take their coins to the nearest arcade to play the classic maze chase. Take a left at the pinball machine and continue straight past the air hockey, following the unmistakable soundtrack of PAC-MAN gobbling dots and avoiding ghosts Inky, Pinky, Blinky and Clyde.

In 1981 alone, Americans inserted billions of quarters to play 75,000 hours of coin-operated games like PAC-MAN. Over the decades since, the hit game has seen versions for PCs, gaming consoles and cell phones.

NVIDIA Researcher Seung-Wook Kim
Game Changer: NVIDIA Researcher Seung-Wook Kim and his collaborators trained GameGAN on 50,000 episodes of PAC-MAN.

The GameGAN edition relies on neural networks, instead of a traditional game engine, to generate PAC-MAN’s environment. The AI keeps track of the virtual world, remembering what’s already been generated to maintain visual consistency from frame to frame.

No matter the game, the GAN can learn its rules simply by ingesting screen recordings and agent keystrokes from past gameplay. Game developers could use such a tool to automatically design new level layouts for existing games, using screenplay from the original levels as training data.

With data from BANDAI NAMCO Research, Kim and his collaborators at the NVIDIA AI Research Lab in Toronto used NVIDIA DGX systems to train the neural networks on the PAC-MAN episodes (a few million frames, in total) paired with data on the keystrokes of an AI agent playing the game.

The trained GameGAN model then generates static elements of the environment, like a consistent maze shape, dots and Power Pellets — plus moving elements like the enemy ghosts and PAC-MAN itself.

It learns key rules of the game, both simple and complex. Just like in the original game, PAC-MAN can’t walk through the maze walls. He eats up dots as he moves around, and when he consumes a Power Pellet, the ghosts turn blue and flee. When PAC-MAN exits the maze from one side, he’s teleported to the opposite end. If he runs into a ghost, the screen flashes and the game ends.

Since the model can disentangle the background from the moving characters, it’s possible to recast the game to take place in an outdoor hedge maze, or swap out PAC-MAN for your favorite emoji. Developers could use this capability to experiment with new character ideas or game themes.

It’s Not Just About Games

Autonomous robots are typically trained in a simulator, where the AI can learn the rules of an environment before interacting with objects in the real world. Creating a simulator is a time-consuming process for developers, who must code rules about how objects interact with one another and how light works within the environment.

Simulators are used to develop autonomous machines of all kinds, such as warehouse robots learning how to grasp and move objects around, or delivery robots that must navigate sidewalks to transport food or medicine.

GameGAN introduces the possibility that the work of writing a simulator for tasks like these could one day be replaced by simply training a neural network.

Suppose you install a camera on a car. It can record what the road environment looks like or what the driver is doing, like turning the steering wheel or hitting the accelerator. This data could be used to train a deep learning model that can predict what would happen in the real world if a human driver — or an autonomous car — took an action like slamming the brakes.

“We could eventually have an AI that can learn to mimic the rules of driving, the laws of physics, just by watching videos and seeing agents take actions in an environment,” said Sanja Fidler, director of NVIDIA’s Toronto research lab. “GameGAN is the first step toward that.”

NVIDIA Research has more than 200 scientists around the globe, focused on areas such as AI, computer vision, self-driving cars, robotics and graphics.

GameGAN is authored by Fidler, Kim, NVIDIA researcher Jonah Philion, University of Toronto student Yuhao Zhou and MIT professor Antonio Torralba. The paper will be presented at the prestigious Conference on Computer Vision and Pattern Recognition in June.

PAC-MANTM & ©BANDAI NAMCO Entertainment Inc.

The post 40 Years on, PAC-MAN Recreated with AI by NVIDIA Researchers appeared first on The Official NVIDIA Blog.

Calling AI: Researchers Dial in Machine Learning for 5G

5G researchers from three top institutions have joined NVIDIA in bringing AI to telecom.

The Heinrich Hertz Institute (HHI), the Technical University in Berlin (TU Berlin) and Virginia Tech are collaborating with NVIDIA to harness the power of GPUs for next-generation cellular networks.

The journey began in October at MWC Los Angeles, where NVIDIA and partners announced plans to enable virtual radio access networks (vRANs) for 5G with GPUs.

NVIDIA also debuted Aerial, a software development kit for accelerating vRANs. And partners Ericsson, Microsoft and Red Hat are working with us to deliver 5G at the edge of the network powered by GPUs.

These vRANs will bring cellular network operators the kind of operational efficiencies that cloud service providers already enjoy. Carriers will program network functions in high-level software languages, easing the work of adding new capabilities and deploying capacity where and when it’s needed.

Forging Wireless Ties

Our new research partnerships with HHI, TU Berlin and Virginia Tech will explore multiple ways to accelerate 5G with AI.

They’ll define novel techniques leveraging GPUs that help wireless networks use precious spectrum more efficiently. The work will span research in reinforcement learning and other techniques that build on the product plans announced in October.

HHI is part of Germany’s Fraunhofer Society, a research group founded in 1928 that has a history of pioneering technologies in mobile and optical networking as well as video compression. The collaboration with TU Berlin includes a new 5G test bed with participation from a number of wireless companies in Germany.

“I want to redesign many algorithms in radio access networks (RAN) so we can perform tasks in parallel, and the GPU is a good architecture for this because it exploits massive parallelism,” said Slawomir Stanczak, a professor at TU Berlin and head of HHI’s wireless networking department.

Stanczak’s teams will explore use cases such as adapting AI to deliver improved 5G receivers. “If we are successful, they could offer a breakthrough in dramatically increasing performance and improving spectral efficiency, which is important because spectrum is very expensive,” he said.

In a session for GTC Digital, Stanczak recently described ways to apply AI to the private 5G campus networks which he believes will be market drivers for vRANs. Stanczak chairs a focus group on the use of AI in 5G for the ITU, a leading communications standards group. He’s also author of a widely cited text on the math behind optimizing wireless networks.

Hitting 5G’s Tight Timing Targets

Work at Virginia Tech is led by Tom Hou, a professor of computer engineering whose team specializes in solving some of the most complex and challenging problems in telecom.

His Ph.D. student, Yan Huang, described in a 2018 paper how he used an NVIDIA Quadro P6000 GPU to solve a complex scheduling problem in a tight 100-microsecond window set by the 5G standard. His latest effort cut the time to 60 microseconds using an NVIDIA Tensor Core V100 GPU.

The work “got an enormous response because at that time people using traditional computational techniques would hit roadblocks — no one in the world could solve such a complex problem in 100 microseconds,” said Hou.

“Using GPUs transformed our research group, now we are looking at AI techniques on top of our newly acquired parallel techniques,” he added.

Specifically, Virginia Tech researchers will explore how AI can automatically find and solve in real time thorny problems optimizing 5G networks. For instance, AI could uncover new ways to weave multiple services on a single frequency band, making much better use of spectrum.

“We have found that for some very hard telecom problems, there’s no math formulation, but AI can learn the problem models automatically, enhancing our GPU-based parallel solutions” said Huang.

Groundswell Starts in AI for 5G

Other researchers, including two who presented papers at GTC Digital, are starting to explore the potential for AI in 5G.

Addressing one of 5G’s top challenges, researchers at Arizona State University showed a new method for directing millimeter wave beams, leveraging AI and the ray-tracing features in NVIDIA Turing GPUs.

And Professor Terng-Yin Hsu described a campus network at Taiwan’s National Chiao Tung University that ran a software-defined cellular base station on NVIDIA GPUs.

“We are very much at the beginning, especially in AI for vRAN,” said Stanczak. “In the end, I think we will use hybrid solutions that are driven both by data and domain knowledge.”

Compared to 4G LTE, 5G targets a much broader set of use cases with a much more complex air interface. “AI methods such as machine learning are promising solutions to tackle these challenges,” said Hou of Virginia Tech.

NVIDIA GPUs bring the programming flexibility of CUDA and cuDNN environments and the scalability of multiple GPUs connected on NVLink. That makes them the platform of choice for AI on 5G, he said.

Today we stand at a pivot point in the history of telecom. The traditional principles of wireless signal processing are based on decades-old algorithms. AI and deep learning promise a revolutionary new approach, and NVIDIA’s GPUs are at the heart of it.

The post Calling AI: Researchers Dial in Machine Learning for 5G appeared first on The Official NVIDIA Blog.

Seeing AI to AI: NVIDIA Deepens Ties with Top Research Center

Andreas Dengel wants to get AI into more people’s hands while he continues to advance the technology.

Sharing that mission and a history of close ties, NVIDIA just joined him and his roughly 1,000 colleagues as a shareholder in the German Research Center for Artificial Intelligence (DFKI).

“A study last week said many companies are collecting data, but they don’t know what to do with it. We can help them join an increasingly data-driven economy,” said Dengel. He serves as head of DFKI’s site in Kaiserslautern in southwest Germany, a member of DFKI’s management board and the scientific director of its smart data and knowledge services group.

One company is already testing a prototype AI sandbox and catalog DFKI built to let users try deep learning. The catalog includes 35 top neural networks for imaging with audio and video models on the way, targeting commercial use next year.

“Removing the boundaries for companies who want to use AI is very important,” said Dengel.

Advancing AI in Germany and Beyond

While DFKI spreads AI’s use, it also aims to advance the technology. The research institute is part of a group making a proposal for a German national supercomputing center focused on AI. A decision is expected later this year on the center, expected to have a budget of up to $16 million a year.

The effort comes amid a pan-European drive to beef up AI research. In mid-March, the European Commission granted a new AI research alliance about $54 million. It’s seen as a downpayment on the region’s future investments in AI.

In this environment, DFKI has no shortage of ambitions. One of its research teams is wrestling with the grand challenge of explainable AI, understanding how deep learning gets its amazing results.

A member of the team presented a paper in June 2018 that won an award from NVIDIA’s founder and CEO Jensen Huang. The paper described a way one neural network can monitor another to understand and optimize its processes.

The work put some light on how deep learning gets its impressive results. But there’s much more to be done as the types of neural networks and datasets proliferate.

“Experts who depend on AI systems should be able to visualize or explain their processes. That’s especially critical for applications on finance and healthcare,” Dengel said.

It’s one of some 250 projects across 20 departments at DFKI, one of the world’s largest AI research centers.

Among other projects, one team is helping the German federal bank apply AI. Another conducts 600-hour tests of car engines, making predictions with AI based on the results. Yet another uses GPUs to analyze high-resolution satellite images, helping coordinate disaster relief efforts.

17 Petaflops of GPU Compute and Growing

DFKI computer rooms pack an estimated 17 petaflops of GPU computing power with multiple NVIDIA DGX systems. They include what was the first DGX-2 in Europe, all linked on Mellanox InfiniBand switches.

It’s a lot of horsepower, but not enough to meet rising demands. The group spun up a climate modeling application two months ago, satellites are “growing exponentially” for imaging applications. And DFKI has a new collaboration with the European Space Agency that will spawn multiple projects.

“We are at the limit of our systems’ use. Our goal is to expand in a big way. We want to provide infrastructure that’s a platform for both the German and the broader European industry,” said Dengel.

“Our experience has shown that by putting apps on NVIDIA GPU clusters companies better understand what GPU acceleration can do for them,” he added.

In the wake of the shareholder agreement, DFKI and NVIDIA are discussing plans for collaborating on software projects. It’s another step in deepening ties at many levels.

The two groups also sometimes share talented people. A former DFKI professor is now an NVIDIA architect, and a handful of DFKI grad students are just back from a sabbatical working with NVIDIA.

The post Seeing AI to AI: NVIDIA Deepens Ties with Top Research Center appeared first on The Official NVIDIA Blog.

Researchers Make Movies of the Brain with CUDA

When colleagues told Sally Epstein they sped up image processing by three orders of magnitude for a client’s brain-on-a-chip technology, she responded like any trained scientist. Go back and check your work, the biomedical engineering Ph.D. told them.

Yet it was true. The handful of researchers at Cambridge Consultants had devised a basket of techniques to process an image on GPUs in an NVIDIA DGX-1 system in 300 milliseconds, a 3,000x boost over the 18 minutes the task took on an Intel Core i9 CPU.

The achievement makes it possible for researchers to essentially watch movies of neurons firing in real time using the brain-on-a-chip technology from NETRI, a French startup.

“Animal studies revolutionized medicine. This is the next step in testing for areas like discovering new drugs,” said Epstein, head of Strategic Technology at Cambridge Consultants, which develops products and technologies for a wide variety of established companies and startups such as NETRI.

The startup designs chips that sport 3D microfluidic channels to host neural tissues and a CMOS camera sensor with polarizing filters to detect individual neurons firing. It hopes its precision imaging can speed the development of novel treatments for neurological disorders such as Alzheimer’s disease.

Facing a Computational Bottleneck

NETRI’s chips generate 100-megapixel images at up to 1,000 frames per second — the equivalent of a hundred 4K gaming systems running at 120fps. Besides spawning tons of data, they use highly complex math.

As a result, processing a single second of a recording took NETRI 12 days, an unacceptable delay. So, the startup turned to Cambridge Consultants to bust through the bottleneck.

“Our track record in scientific and biological imaging turned out to be very relevant,” said Monty Barlow, Director of Strategic Technology at Cambridge Consultants. And when NETRI heard about the 3,000x boost, “they trusted us even though we didn’t trust ourselves at first,” he quipped.

Leveraging Math, Algorithms and GPUs

A handful of specialists at Cambridge Consultants delivered the 3,000x speedup using multiple techniques. For example, math and algorithm experts employed a mix of Gaussian filters, multivariate calculus and other tools to eliminate redundant tasks and reduce peak RAM requirements.

Software developers migrated NETRI’s Python code to CuPy to take advantage of the massive parallelism of NVIDIA’s CUDA software. And hardware specialists optimized the code to fit into GPU memory, eliminating unnecessary data transfers inside the DGX-1.

The CUDA profiler helped find bottlenecks in NETRI’s code and alternatives to resolve them. “NVIDIA gave us the tools to execute this work efficiently — it happened within a week with a core team of four researchers and a few specialists,” said Epstein.

Looking ahead, Cambridge Consultants expects to find further speedups for the code using the DGX-1 that could enable real-time manipulation of neurons using a laser. Researchers also aim to explore NVIDIA IndeX software to help visualize neural activity.

The work with NETRI is one of several DGX-1 applications at the company. It also hosts a Digital Greenhouse for AI research. Last year, it used the DGX-1 to create a low-cost but highly accurate tool for monitoring tuberculosis.

The post Researchers Make Movies of the Brain with CUDA appeared first on The Official NVIDIA Blog.

NVIDIA Gives COVID-19 Researchers Free Access to Parabricks

When a crisis hits, we all pitch in with what we have. In response to the current pandemic, NVIDIA is sharing tools with researchers that can accelerate their race to understand the novel coronavirus and help inform a response.

Starting today, NVIDIA will provide a free 90-day license to Parabricks to any researcher in the worldwide effort to fight the novel coronavirus. Based on the well-known Genome Analysis Toolkit, Parabricks uses GPUs to accelerate by as much as 50x the analysis of sequence data.

We recognize this pandemic is evolving, so we’ll monitor the situation and extend the offer as needed.

If you have access to NVIDIA GPUs, fill out this form to request a Parabricks license.

For researchers working with Oxford Nanopore long-read data, a repository of GPU-accelerated tools is available on GitHub. In addition, the following applications already have NVIDIA GPU acceleration built in: Medaka, Racon, Raven, Reticulatus, Unicycler.

Researchers are sequencing both the novel coronavirus and the genomes of people afflicted with COVID-19 to understand, among other things, the spread of the disease and who is most affected. But analyzing genomic sequences takes time and computing muscle.

Accelerating science has long been part of NVIDIA’s core mission. The Parabricks team joined NVIDIA in December, providing the latest tool for that work. It can reduce the time for variant calling on a whole human genome from days to less than an hour on a single server.

Given the unprecedented spread of the pandemic, getting results in hours versus days could have an extraordinary impact on understanding the virus’s evolution and the development of vaccines.

NVIDIA is inviting our family of partners to join us in matching this urgent effort to assist the research community. We’re in discussions with cloud service providers and supercomputing centers to provide compute resources and access to Parabricks on their platforms.

We’ll update this blog with links to others who can provide cloud-based access to NVIDIA GPUs and this software as those sources become available.

Support links:

The post NVIDIA Gives COVID-19 Researchers Free Access to Parabricks appeared first on The Official NVIDIA Blog.

Italy Forges AI Future in Partnership with NVIDIA

Italy is well known for its architecture, culture and cuisine. Soon, its contributions to AI may be just as renowned.

Taking a step in that direction, a national research organization forged a three-year collaboration with NVIDIA. Together they aim to accelerate AI research and commercial adoption across the country.

Leading the charge for Italy is CINI, the National Inter-University Consortium for Informatics that includes a faculty of more than 1,300 professors in various computing fields across 39 public universities.

CINI’s National Laboratory of Artificial Intelligence and Intelligence Systems (AIIS) is spearheading the effort as part of its goal to expand Italy’s ecosystem for both academic research and commercial applications of AI.

“Leveraging NVIDIA’s expertise to build systems specifically designed for the creation of AI will help secure Italy’s position as a top player in AI education, research and industry adoption,” said Rita Cucchiara, a professor of computer engineering and science and director of AIIS.

National effort begins in Modena

The joint initiative aims to train students, nurture startups and spread adoption of the attest AI technology throughout Italy. As a first step, the partners will create a local hub at the University of Modena and Reggio Emilia (Unimore) for the global NVIDIA AI Technology Center.

The partnership marks an important expansion of NVIDIA’s work with the university whose roots date back to the medieval period.

In December, the company supported research on a novel way to automate the process of describing actions in a video. A team of four researchers at Unimore and one from Milan-based AI startup Metaliquid developed an AI model that achieved up to a 16 percent relative improvement compared to prior solutions. In a final stage of the project, NVIDIA helped researchers analyze their network’s topology to optimize training it on an NVIDIA DGX-1 system.

In July, Unimore and NVIDIA collaborated on an event for AI startups. Unimore’s AImageLab hosted the event, which included representatives of NVIDIA’s Inception program, an initiative to nurture AI startupswith access to the company’s technology and ecosystem.

The collaboration comes at a time when the AImageLab, host for the new NVIDIA hub, is already making its mark in areas such as machine vision and medical imaging.

Winning kudos in image recognition

In September, two world-class research events singled out the AImageLab for recognition. One team from the lab won a best paper award at the International Conference on Computer Analysis of Images and Patterns. Another came third out of 64 research groups in an international competition using AI to classify skin lesions.

The Modena hub becomes the latest of more than 12 collaborations with countries worldwide for the NVIDIA AI Technology Center. NVAITC maintains an open database of research and tools developed with and for its partners.

Overall, the new collaboration “will bring together NVIDIA and CINI in our shared mission to enable, support and inform Italy’s AI ecosystem for research, industry and society,” said Simon See, senior director of NVAITC.

The post Italy Forges AI Future in Partnership with NVIDIA appeared first on The Official NVIDIA Blog.

Life Observed: Nobel Winner Sees Biology’s Future with GPUs

Five years ago, when Eric Betzig got the call he won a Nobel Prize for inventing a microscope that could see features as small as 20 nanometers, he was already working on a new one.

The new device captures the equivalent of 3D video of living cells — and now it’s using NVIDIA GPUs and software to see the results.

Betzig’s collaborator at the University of California at Berkeley, Srigokul Upadhyayula (aka Gokul), helped refine the so-called Lattice Light Sheet Microscopy (LLSM) system. It generated 600 terabytes of data while exploring part of the visual cortex of a mouse in work published earlier this year in Science magazine. A 1.3TB slice of that effort was on display at NVIDIA’s booth at last week’s SC19 supercomputing show.

Attendees got a glimpse of how tomorrow’s scientists may unravel medical mysteries. Researchers, for example, can use LLSM to watch how protein coverings on nerve axons degrade as diseases such as muscular sclerosis take hold.

Future of Biology: Direct Visualization

“It’s our belief we will never understand complex living systems by breaking them into parts,” Betzig said of methods such as biochemistry and genomics. “Only optical microscopes can look at living systems and gather information we need to truly understand the dynamics of life, the mobility of cells and tissues, how cancer cells migrate. These are things we can now directly observe.

“The future of biology is direct visualization of living things rather than piecing together information gleaned by very indirect means,” he added.

It Takes a Cluster — and More

Such work comes with heavy computing demands. Generating the 600TB dataset for the Science paper “monopolized our institution’s computing cluster for days and weeks,” said Betzig.

“These microscopes produce beautifully rich data we often cannot visualize because the vast majority of it sits in hard drives, completely useless,” he said. “With NVIDIA, we are finding ways to start looking at it.”

The SC19 demo — a multi-channel visualization of a preserved slice of mouse cortex — ran remotely on six NVIDIA DGX-1 servers, each packing eight NVIDIA V100 Tensor Core GPUs. The systems are part of an NVIDIA SATURNV cluster located near its headquarters in Santa Clara, Calif.

Berkeley researchers gave SC19 attendees a look inside the visual cortex of a mouse — visualized using NVIDIA IndeX.

The key ingredient for the demo and future visualizations is NVIDIA IndeX software, an SDK that allows scientists and researchers to see and interact in real time with massive 3D datasets.

Version 2.1 of IndeX debuted at SC19, sporting a host of new features, including GPUDirect Storage, as well as support for Arm and IBM POWER9 processors.

After seeing their first demos of what IndeX can do, the research team installed it on a cluster at UC Berkeley that uses a dozen NVIDIA TITAN RTX and four V100 Tensor Core GPUs. “We could see this had incredible potential,” Gokul said.

Closing a Big-Data Gap

The horizon holds plenty of mountains to climb. The Lattice scope generates as much as 3TB of data an hour, so visualizations are still often done on data that must be laboriously pre-processed and saved offline.

“In a perfect world, we’d have all the information for analysis as we get the data from the scope, not a month or six months later,” said Gokul. The time between collecting and visualizing data can stretch from weeks to months, but “we need to tune parameters to react to data as we’re collecting it” to make the scope truly useful for biologists, he added.

NVIDIA IndeX software, running on its increasingly powerful GPUs, helps narrow that gap.

In the future, the team aims to apply the latest deep learning techniques, but this too presents heady challenges. “There are no robust AI models to deploy for this work today,” Gokul said.

Making the data available to AI specialists who could craft AI models would require shipping crates of hard drives on an airplane, a slow and expensive proposition. That’s because the most recent work produced over half a petabyte of data, but cloud services often limit uploads and downloads to a terabyte or so per day.

Betzig and Gokul are talking with researchers at cloud giants about new options, and they’re exploring new ways to leverage the power of GPUs because the potential of their work is so great.

Coping with Ups and Downs

“Humans are visual animals,” said Betzig. “When most people I know think about a hypothesis, they create mental visual models.

“The beautiful thing about microscopy is you can take a model in your head with all its biases and immediately compare it to the reality of living biological images. This capability already has and will continue to reveal surprises,” he said.

The work brings big ups and downs. Winning a Nobel Prize “was a shock,” Betzig said. “It kind of felt like getting hit by a bus. You feel like your life is settled and then something happens to change you in ways you wouldn’t expect — it has good and bad sides to it.”

Likewise, “in the last several years working with Gokul, every microscope had its limits that led us to the next one. You take five or six steps up to a plateau of success and then there is a disappointment,” he said.

In the partnership with NVIDIA, “we get to learn what we may have missed,” he added. “It’s a chance for us to reassess things, to understand the GPU from folks who designed the architecture, to see how we can merge our problem sets with new solutions,” he said.

Note: The picture at top shows Berkeley researchers Eric Betzig, Ruixian Gao and Srigokul Upadhyayula with the Lattice Light Sheet microscope.

The post Life Observed: Nobel Winner Sees Biology’s Future with GPUs appeared first on The Official NVIDIA Blog.

AI’s Latest Adventure Turns Pets into GANimals

Imagine your Labrador’s smile on a lion or your feline’s finicky smirk on a tiger. Such a leap is easy for humans to perform, with our memories full of images. But the same task has been a tough challenge for computers — until the GANimal.

A team of NVIDIA researchers has defined new AI techniques that give computers enough smarts to see a picture of one animal and recreate its expression and pose on the face of any other creature. The work is powered in part by generative adversarial networks (GANs), an emerging AI technique that pits one neural network against another.

You can try it for yourself with the GANimal app. Input an image of your dog or cat and see its expression and pose reflected on dozens of breeds and species from an African hunting dog and Egyptian cat to a Shih-Tzu, snow leopard or sloth bear.

I tried it, using a picture of my son’s dog, Duke, a mixed-breed mutt who resembles a Golden Lab. My fave — a dark-eyed lynx wearing Duke’s dorky smile.

There’s potential for serious applications, too. Someday movie makers may video dogs doing stunts and use AI to map their movements onto, say, less tractable tigers.

The team reports its work this week in a paper at the International Conference on Computer Vision (ICCV) in Seoul. The event is one of three seminal conferences for researchers in the field of computer vision.

Their paper describes what the researchers call FUNIT, “a Few-shot, UNsupervised Image-to-image Translation algorithm that works on previously unseen target classes that are specified, at test time, only by a few example images.”

“Most GAN-based image translation networks are trained to solve a single task. For example, translate horses to zebras,” said Ming-Yu Liu, a lead computer-vision researcher on the NVIDIA team behind FUNIT.

“In this case, we train a network to jointly solve many translation tasks where each task is about translating a random source animal to a random target animal by leveraging a few example images of the target animal,” Liu explained. “Through practicing solving different translation tasks, eventually the network learns to generalize to translate known animals to previously unseen animals.”

Before this work, network models for image translation had to be trained using many images of the target animal. Now, one picture of Rover does the trick, in part thanks to a training function that includes many different image translation tasks the team adds to the GAN process.

The work is the next step in Liu’s overarching goal of finding ways to code human-like imagination into neural networks. “This is how we make progress in technology and society by solving new kinds of problems,” said Liu.

The team — which includes seven of NVIDIA’s more than 200 researchers — wants to expand the new FUNIT tool to include more kinds of images at higher resolutions. They are already testing it with images of flowers and food.

Liu’s work in GANs hit the spotlight earlier this year with GauGAN, an AI tool that turns anyone’s doodles into photorealistic works of art.

The GauGAN tool has already been used to create more than a million images. Try it for yourself on the AI Playground.

At the ICCV event, Liu will present a total of four papers in three talks and one poster session. He’ll also chair a paper session and present at a tutorial on how to program the Tensor Cores in NVIDIA’s latest GPUs.

The post AI’s Latest Adventure Turns Pets into GANimals appeared first on The Official NVIDIA Blog.

Pixel-Perfect Perception: How AI Helps Autonomous Vehicles See Outside the Box

Editor’s note: This is the latest post in our NVIDIA DRIVE Labs series, which takes an engineering-focused look at individual autonomous vehicle challenges and how NVIDIA DRIVE addresses them. Catch up on all of our automotive posts, here.

A self-driving car’s view of the world often includes bounding boxes — cars, pedestrians and stop signs neatly encased in red and green rectangles.

In the real world, however, not everything fits in a box.

For highly complex driving scenarios, such as a construction zone marked by traffic cones, a sofa chair or other road debris in the middle of the highway, or a pedestrian unloading a moving van with cargo sticking out the back, it’s helpful for the vehicle’s perception software to provide a more detailed understanding of its surroundings.

Such fine-grained results can be obtained by segmenting image content with pixel-level accuracy, an approach known as panoptic segmentation.

With panoptic segmentation, the image can be accurately parsed for both semantic content (which pixels represent cars vs. pedestrians vs. drivable space), as well as instance content (which pixels represent the same car vs. different car objects).

Planning and control modules can use panoptic segmentation results from the perception system to better inform autonomous driving decisions. For example, the detailed object shape and silhouette information helps improve object tracking, resulting in a more accurate input for both steering and acceleration. It can also be used in conjunction with dense (pixel-level) distance-to-object estimation methods to help enable high-resolution 3D depth estimation of a scene.

Single DNN Approach

NVIDIA’s approach achieves pixel-level semantic and instance segmentation of a camera image using a single, multi-task learning deep neural network. This approach enables us to train a panoptic segmentation DNN that understands the scene as a whole versus piecewise.

It’s also efficient. Just one end-to-end DNN can extract all this rich perception information while achieving per-frame inference times of approximately 5ms on our embedded in-car NVIDIA DRIVE AGX platform. This is much faster than state-of-the-art segmentation methods.

DRIVE AGX makes it possible to simultaneously run the panoptic segmentation DNN along with many other DNN networks and perception functions, localization, and planning and control software in real time.

Panoptic segmentation DNN output from in-car inference on embedded AGX platform. Top: predicted objects and object classes (blue = cars; green = drivable space; red = pedestrians). Bottom: predicted object-class instances along with computed bounding boxes (shown in different colors and instance IDs).

As shown above, the DNN is able to segment a scene into several object classes, as well as detect different instances of these object classes, as shown with the unique colors and numbers in the bottom panel.

On-Point Training and Perception

The rich pixel-level information provided by each frame also reduces training data volume requirements. Specifically, because more pixels per training image represent useful information, the DNN is able to learn using fewer training images.

Moreover, based on the pixel-level detection results and post-processing, we’re also able to compute the bounding box for each object detection. All the perception advantages offered by pixel-level segmentations require processing, which is why we developed the powerful NVIDIA DRIVE AGX Xavier.

As a result, the pixel-level details panoptic segmentation provides make it possible to better perceive the visual richness of the real world in support of safe and reliable autonomous driving.

The post Pixel-Perfect Perception: How AI Helps Autonomous Vehicles See Outside the Box appeared first on The Official NVIDIA Blog.