Putting Their Foot Down: GOAT Uses AI to Stomp Out Fake Air Jordan and Adidas Yeezy Sneakers

Sneaker aficionados invest hundreds of dollars into rare Nike Air Jordans and the hottest Kanye West Adidas Yeezys. But scoring an authentic pair amid a crush of counterfeits is no slam dunk.

Culver City, Calif., startup GOAT (a nod to the sports shorthand for “greatest of all time”) operates the world’s largest sneaker marketplace that uses AI to stomp out fakes. The company offers a seal of authenticity for shoes approved for sale on its site.

Counterfeit sneakers are rampant online for some of the most sought after basketball brands.

“Yeezys and Jordans are now the most faked shoes in the world, and over 10 percent of all sneakers sold online are fake,” said Michael Hall, director of data at GOAT.

A pair of sought-after Kanye West Adidas Yeezys or Nike Air Jordans can easily set you back more than $300.

Pop culture interest in iconic shoes developed for sports stars and celebrity rappers is fueling instant sellouts in new releases. Meanwhile, there’s a heated aftermarket for the most popular footwear fashions as well as scarce vintage and retro models.

As a result, sneaker fans and novices alike are turning to a new wave of shoe sellers, such as GOAT, to ensure they’re getting getting an authentic pair of the most sought-after shoes.

GOAT pioneered the ship-to-verify model in the sneaker industry. This means that sellers can list any shoes on GOAT’s marketplace, but shoes that sell are first sent to the company for authentication by its image detection AI. If the shoes are found to be replicas or not as described, they don’t ship and buyers are given a refund.

Founded in 2015, GOAT’s business is booming. The startup, which has expanded to more than 500 employees, attracts more than 10 million users and has the largest catalog of sneakers in the world at 35,000 skus. This year, the company merged with Flight Club, a sneaker consignment store with locations in Los Angeles,New York City and Miami.

GOAT’s popular app and website — some users have sold more than $10 million in sneakers — has secured nearly $100 million in venture capital funding. The company is a member of NVIDIA’s Inception program, which offers technical guidance to promising AI startups.

AI to Kick Out Counterfeits

When you’re offering 35,000 unique styles, tracking down counterfeit sneakers is no small challenge. GOAT has teams of sneaker experts trained in the art of spotting replicas without AI. “They can spot a fake in like 10 seconds,” said Emmanuelle Fuentes, lead data scientist at GOAT.

Image recognition assists GOAT’s teams of authenticators and quality assurance representatives to ID and authenticate shoes in the warehouse. And the more GOAT’s experts provide helpful metadata to train the AI as they work, the better it helps all those vetting sneakers.

There’s a long list of data signals that are fed into a cloud instance of GPUs for the identification process and for training the network. GOAT’s convolutional neural networks are trained for anomaly and fraud detection.

GOAT, which has multiple neural networks dedicated to brands of sneakers, provides proprietary tools to help its authenticators upload data to train its identification networks.

GPUs Slam Dunk

Tracking and sharing expertise on so many models of high-end sneakers requires logging a ton of photos of authentic sneakers to help assist team members in handling shoes sent in for verification.

“The resolution that we are capturing things at and the scale that we are capturing the images — it’s a high-resolution, massive computational challenge requiring GPUs,” said Fuentes.

GOAT turned to NVIDIA TITAN Xp GPUs and NVIDIA Tesla GPUs on P2 instances of AWS running the cuDNN-accelerated PyTorch deep learning framework to initially train their neural networks on 75,000 images of authentic sneakers.

The company relies on the power of GPUs for identification of all of its sneaker models, Hall added. “For some of the most-coveted sneakers, there are more inauthentic pairs than real ones out there. Previously there wasn’t a way for sneakerheads to purchase with confidence,” he said.

The post Putting Their Foot Down: GOAT Uses AI to Stomp Out Fake Air Jordan and Adidas Yeezy Sneakers appeared first on The Official NVIDIA Blog.

AI Makes a Splash: Parisian Trio Navigates Autonomous Cargo Ships

It started as friends hacking a radio-controlled boat over a weekend for fun. What happened next is classic Silicon Valley: Three childhood buddies parlay robotics and autonomous vehicle skills into an autonomous ship startup and cold-call the world’s third-largest cargo shipper.

San Francisco-based Shone — founded by Parisian classmates Ugo Vollmer, Clement Renault and Antoine de Maleprade — has made a splash in maritime and startup circles. The company landed a pilot deal with shipping giant CMA CGM and recently won industry recognition.

Left to right: Shone co-founders Antoine de Maleprade, Ugo Vollmer and Clement Renault.

Shone aims to modernize shipping. The startup applies NVIDIA GPUs to a flood of traditional cargo ship data such as sonar, radar, GPS and AIS, a ship-to-ship tracking system. This has enabled it to quickly process terabytes of training data on its custom algorithms to develop perception, navigation and control for ocean freighters. The company has added cameras to offer better seafaring object detection as well.

“The first part is packaging all of the perception so the crew can make better decisions,” said Vollmer, the company’s CEO, previously an autonomous vehicle maps engineer at Mapbox. “But there’s tons of value of connecting communications from the ship to the shore.”

Cargo ships are among a wave of industries, including locomotives, joining the autonomous vehicle revolution.

Shone is a member of NVIDIA Inception, a virtual accelerator program that helps startups get to market faster.

GPUs Set Sail

Founded in 2017, Shone has expanded to eight employees to navigate seafaring AI. Its NVIDIA GPU-powered software is now deployed on several CMA CGM cargo ships in pilot tests to help with perception for ship captains.

“What is particularly interesting for CMA CGM is what artificial intelligence can bring to systems on board container ships in terms of safety. AI will facilitate the work of crews on board, whether in decision support, maritime safety or piloting assistance,” said Jean-Baptiste Boutillier, deputy vice president at CMA CGM.

It didn’t come easy. The trio of scrappy entrepreneurs had to hustle. After hacking the radio-controlled boat and realizing they had a model for an autonomous boat, they raised $200,000 in friends-and-family money to start the company.

Next they got accepted into Y Combinator. Shone’s partnership with CMA CGM, which operates 500 ships worldwide, came as the team was working as a member of the winter 2018 batch at the accelerator and urgently seeking a pilot to prove their startup’s potential.

Renault and Vollmer recall cold-calling CMA CGM, the surprise of getting an appointment, and going in jeans and T-shirts and underprepared to meet with executives — who were in suits. Despite that, the execs were impressed with the team’s technical knowledge and encouraged them to come back — in six months.

“There is this industry that is moving 90 percent of the products in the world, and the tech is basically from the ‘80s — we were like, ‘What? How is that possible?’” said Renault, who previously worked on autonomous trucks at Starsky Robotics.

Prototype AI Boat

Not to be discouraged by the first meeting, the team decided to buy a real boat to outfit with AI to show CMA CGM just how serious they were. Renault spent $10,000 for a 30-foot boat he located on Craigslist, but the trailer blew a tire on the way back from the Sacramento River delta. Just one more obstacle to overcome, but he got help and got it back to the office.

De Maleprade’s robotics skills came into play next. (“He was building rockets when he was 14 and blowing things up,” Vollmer said.) De Maleprade fitted the boat with a robotic hydraulic steering mechanism. The other two went to work on the software with him, installing a GPU on the boat, as well.

The boat development was sped by training their algorithms on GPU workstations, and the on-board GPUs of the boat enabled inferencing, said de Maleprade.

Three months later, the trio had the autonomous boat prototype ready. They showed off its capabilities to CMA CGM, which was even more impressed. The executives invited them to develop the platform on several of their freighters, which span four football fields in length and transport 14,000 containers from Southern California to China.

“CMA CGM has built its success on strong entrepreneurial values and constant innovations. That is why we decided to allow the Shone team to board some of our ships to develop their ideas and their potential,” said Boutillier.

The founders often sleep on the cargo ships, departing from the Port of Long Beach to arrive in the Port of Oakland, to observe the technology in action and study CMA CGM’s needs.

Synthesizing all the maritime data with various home-brewed algorithms and other techniques to develop Shone’s perception platform was a unique challenge, said de Maleprade, who worked on robotics before becoming CTO at Shone.

“We took a video with a drone to show CMA CGM our prototype boat could offer autonomous navigation assistance for crews. They liked what we had,” said de Maleprade. He says they’re now working on simulation to speed their development timeline.

The Shone team recently finished up at Y Combinator, leaving a strong impression at its Demo Day presentation and with $4 million in seed funding from investors.

The post AI Makes a Splash: Parisian Trio Navigates Autonomous Cargo Ships appeared first on The Official NVIDIA Blog.

SC18: World’s Best-Educated Graffiti Wall Celebrates GPU Developer Community

Call it the heart of the heart of the SC18 supercomputing show.

Possibly the world’s best-educated graffiti wall, the whiteboard-sized graphic tracks the dizzying growth of the NVIDIA Developer Program — from a single soul to its current 1.1 million members. In a rainbow of colors and multiplicity of handwriting styles, developers are penning notes describing their own contributions along the way, corresponding to the year it occurred.

Beside an illuminated line tracking the number of developers each year, towers of note cards are building, growing by the hour as more individuals take in the project. The work of computing legends sits beside those of anonymous engineers.

2008 begins with “World’s First GPU-accelerated Supercomputer in Top500: Tsubame 1.2.” Midway above the 2010 stack is the first “GPU-accelerated Molecular Simulation” by Erik Lindahl, the Stockholm University biologist. 2012 features “Alexnet Wins ImageNet” by Alex Krizhevsky, considered a defining moment ushering in the era of artificial intelligence.

“It’s a crowdsourced celebration of the GPU developer ecosystem,” said Greg Estes, vice president of developer programs and corporate marketing at NVIDIA.

The living yearbook — which after the show will take a pride of place in NVIDIA’s Santa Clara campus — depicts the story of the growth of accelerated computing, propelled less by silicon and more by individual imagination and dazzling coding skills.

It embraces work that’s been awarded Nobel Prizes in physics and chemistry. It’s made possible the world’s fastest supercomputers, which are driving groundbreaking research in fields as far-flung as particle physics and planetary science. And it’s opened the door to video games so realistic that they begin to blur with movies. And to movies with effects so mind blowing, they push to the far edges of human imagination.

The NVIDIA Developer Program, which recently pushed above 1.1 million individuals, continues to grow steadily because of the emergence of AI, as well as continued growth in robotics, game development, data science and ray tracing.

What developers receive when they sign up for the program for free is access to more than 100 SDKs, performance analysis tools, member discounts and hands-on training on CUDA, OpenACC, AI and machine learning through the NVIDIA Deep Learning Institute. It’s a package of tools and offerings that simplifies the task of tapping into the incredible processing power and efficiency of GPUs, the performance of which has increased many times over in just a few generations.

The timeline begins with hoary entries that, by the standards of accelerated computing, seem modest and almost accidental. But they laid the foundation for work to come, including monumental achievements that have changed the shape of science.

Among them, the 2002 milestone when NVIDIA’s Mark Harris coined the term GPGPU — general purpose computation on graphics processing units. This inspired developers to find ways to use GPUs for compute functions beyond their traditional focus on graphics.

Two years later, NVIDIA’s Ian Buck and a team of researchers introduced Brooks for GPUs, the first GPGPU programming language. And two years after that, in 2006, we launched CUDA, our accelerated parallel computing architecture, which has since been downloaded more than 12 million times.

The board’s very first entry, though, far precedes Harris’s and Buck’s work, and was even more foundational. “Just The First,” it reads, signed by Jensen Huang, dated 1993, the year of NVIDIA’s founding.

The post SC18: World’s Best-Educated Graffiti Wall Celebrates GPU Developer Community appeared first on The Official NVIDIA Blog.

AI to Neigh About: Magic AI Trots Out Horse Intelligence

Magic AI is galloping into the internet of horses arena.

The Seattle-based startup, an angel-funded team of five, has been developing AI for stable managers and riders to monitor the health and security of horses from video feeds.

Image recognition has been a boon to agriculture businesses, including those in the cattle industry. Magic AI corrals algorithms for its AI-powered software to monitor video and help better manage horses, streaming the video to its servers for processing.

Magic AI founder and CEO Alexa Anthony knows the needs of horse owners. The daughter of a horse trainer, she grew up riding in the Seattle area and is a former NCAA national champion in horse jumping.

“If you have a Lamborghini, you have it in a garage with an alarm. Horses are often times in a barn in remote places without any security that they are ok when you are sleeping,” she said.

Magic AI’s StableGuard, a system of cameras that works with a mobile app to keep tabs on horses, provides GPU-driven video monitoring and emergency alerts. StableGuard can be configured to recognize riders and staff of stables as well as to send alerts if strangers enter.

Horse Data Hurdle

Building StableGuard wasn’t easy. The developers at Magic AI initially couldn’t find enough publicly available horse images to adequately train its deep neural networks. They began with MXNet and horse images from the classic ImageNet that proved problematic. 

“They actually trained abysmally because the angle of our cameras is overhead, very different than ImageNet,” said Kyle Lampe, vice president of engineering at Magic AI “That really threw off most of the things we used to train.”

The startup enlisted convolutional neural networks and other machine learning and vision techniques as well as some motion analysis, he said.

Magic AI’s developers relied heavily on transfer learning to add to a number of different image classification networks. Lampe said that with enough new data — “terabytes and terabytes” of video images — they were able to successfully build on top of networks that had already been trained and used in competitions.

“When you do transfer learning, you’re putting images in after the fact and it is applying everything that it has learned before,” he said.

Developers at Magic AI relied on GPUs on desktop as well as on AWS to handle the hefty training workloads on the deep neural networks.

Horse Health Results

The original inspiration for Magic AI came when Anthony’s horse died of colic. Colic symptoms are fairly easy to spot — rolling on the ground, kicking at the stomach, pawing on the ground — and can be identified with image classification algorithms.

Today, Magic AI is adding to a growing list of health indicators for customers to track using its StableGuard system. StableGuard enables customers to keep track of how often horses are eating and drinking, on their feet, and whether they are blanketed when it’s cold, offering more ways to support horse wellness.

The company can also alert horse owners to signs that an animal is close to giving birth. “We can see signs that are indicative of birth. And then you can look at the live feed on your phone,” said Anthony.

Magic AI has a pilot customer, Thunderbird Show Park, in British Columbia, Canada. On that site, StableGuard is in 120 horse stalls. It offers the horse-monitoring service for $15 a day to those there for horse-jumping tournaments and other events.

Most of these sites are powered by a GPU on site, sizable hard drive storage and other computing resources to run Magic AI’s service. “I am excited to see how this technology can improve the wellness of animals globally,” said Anthony.



The post AI to Neigh About: Magic AI Trots Out Horse Intelligence appeared first on The Official NVIDIA Blog.

Pixellot Scores a Goal for Amateur Sports by Automating Video Analysis with AI

Stephen Curry. Cristiano Ronaldo. Tom Brady. These future hall of famers and their pro teammates all up their game using extensive video analysis. Why should it be any different for weekend warriors and aspiring rec league stars?

Startup Pixellot has a way to offer everyone the chance to get some game, whether it’s on the court, the field or the mat.

Pixellot, based near Tel Aviv, develops AI for automated video production cameras that capture sports action, providing streaming of amateur games, coaching tools and easily shared clips.

The company is among a cadre of upstarts that enlist multiple cameras and AI to autonomously capture panoramic views and track the ball and key moments in games. Copenhagen-based Veo is working on AI cameras for amateur soccer.

Founded in 2013, Pixellot — which recently raised a $30 million Series B round of financing — claims bragging rights as the largest source of automated sports video production.

The startup produces nearly 30,000 hours of sports content a month and has sold more than 2,500 cameras packing GPUs and its software worldwide.

Customers include sports leagues, clubs and coaches across the U.S., Europe, Asia and Latin America. The company’s video-coaching features are used by soccer leagues such as the English Premier league, La Liga, LigaMX and the Bundesliga.

“Pixellot is the world leader today in automatic sports production. We cater to the entire football vertical in Mexico, for example, where it’s almost a registered religion,” said Gal Oz, CTO and founder of Pixellot.

Pixellot aims to bring low-cost video production to all amateur sports. Customers include those for basketball, football, soccer, ice hockey, volleyball, handball, lacrosse, baseball and field hockey..

Coaching with AI

Pixellot’s system enables a single person to edit and broadcast a video stream of a sports event in real time. This allows for semi-professional leagues, colleges and others to produce and offer low-cost broadcasts for video streaming. Customers can use its software suite to see multiple views of video — including the center of action, presets on a specific part of the field  and replays — and mix it in real time for streaming.

Coaches use it to quickly produce video clips. Key plays can be tagged for review at a later time, and the editing suite enables coaches to review each scene, frame by frame, and cut clips to share with coaches and players.

“We collect the highlights of the game based on some deep learning processes. We know how to collect the main events and to use them after the game for a highlight clip,” said Oz.

Score for AI

Pixellot’s NVIDIA GPU-powered multi-camera systems feature as many as four cameras and capture video up to 8K. They’re intended to be installed in a fixed location for use without a camera operator. The systems can be used individually or combined, if needed, to produce stitched panoramic video to cover an entire field of action.

Pixellot’s algorithms — convolutional neural networks for image recognition — enable its software to follow the game for automatic video production and to capture highlights. In soccer, for instance, it knows that kicks on the goal are just about as important to focus on as scored goals themselves.

The startup has amassed hundred of thousands of hours of sports videos to train its algorithms to catch key moments. The company trained its algorithms on NVIDIA GPUs on local workstations.

The post Pixellot Scores a Goal for Amateur Sports by Automating Video Analysis with AI appeared first on The Official NVIDIA Blog.

OmniSci (formerly MapD) Charts $55M in Funding for GPU-Powered Analytics

OmniSci, a data-visualization startup that’s just changed its name from MapD, has a chart of its own: hockey stick growth.

The pioneer in GPU-driven analytics, which delivers its popular data visualizations in the blink of an eye, on Wednesday landed $55 million in Series C funding from investors, including NVIDIA. It was the fourth time NVIDIA has participated in one of its fund-raising rounds.

OmniSci CEO Todd Mostak, who originally built the technology as a researcher at Harvard and MIT, realized early the speed advantages of GPUs over CPUs to query and visualize massive datasets.

OmniSci’s SQL database engine fully exploits GPUs, offering in-memory access to big data. Its software allows customers to slice and dice data and serve up graphics and visualizations from billions of data points on the fly. It quickly made a splash for its ability to power real-time visual data analytics over more than a billion tweets.

“We cache as much as possible on the memory of these GPUs,” said Mostak, the company’s CEO. “We have taken the fastest hardware out there to optimize our software. You have all this legacy software out there that relies on CPUs, and it simply cannot provide interactive and real-time analytics at the scales data-driven organizations are grappling with.”

The startup’s software is aimed at data-intensive sectors, including automotive, telecommunications, financial, entertainment, defense and intelligence. The company recently rebranded to OmniSci, inspired by the idea of the endless pursuit of knowledge and insight for everyone.

OmniSci unleashes the massively parallel processing capabilities of GPUs to instantly query multibillion-sized datasets. Customers use OmniSci to answer queries in milliseconds that, in some cases, used to take nearly a day. And OmniSci can exploit the visual computing benefits of GPUs to transform massive datasets into interactive visualizations.

A screenshot of an OmniSci public demo, showcasing interactive cross-filtering and drill-down on nearly 12B rows of telematics data from U.S. ship transponders.

Lightning-fast analytics from OmniSci are powering faster, and financially impactful, business decisions. Verizon, for example, uses OmniSci to process billions of rows of communications data in real time to monitor network performance in real time, improving customer service and optimizing field service decisions.

“Analytics and data science professionals are realizing there’s this new architecture emerging into the mainstream,” said Mostak.

OmniSci can be installed on-premises, and it runs on AWS and Google Cloud, harnessing NVIDIA GPUs. In March, the startup launched its own GPU-accelerated analytics-as-a-service cloud at our GPU Technology Conference. OmniSci Cloud makes the world’s fastest open source SQL engine and visual analytics software available in under 60 seconds, from a web browser.

OmniSci plans to use the funding to accelerate research and development as well as to support the open source community and expand to meet “rapidly growing enterprise demand,” particularly in the U.S. federal market and Europe, Mostak said.

The company has experienced explosive growth, significantly expanding its employee base and more than tripling customers in past year alone, according to the company.

Lead investor Tiger Global Management cites OmniSci’s potential for major impacts to the analytics market, noting a “growing ecosystem of software purpose-built to run on GPUs can have a transformative impact to a number of software categories,” according to partner Lee Fixel.

OmniSci, founded in 2013, has now raised $92 million in total funding. See its technology in action at GTC Europe, which runs Oct. 9 to 11.

The post OmniSci (formerly MapD) Charts $55M in Funding for GPU-Powered Analytics appeared first on The Official NVIDIA Blog.

Robot Tamer Madeline Gannon: New Platform Will Bring Machines to Heel at Scale

Training, testing and coding robots is a grueling process. Our recently launched Isaac platform promises to change all that.

Few know that better than roboticist Madeline Gannon. For the past month, she’s been hard at work developing a robotic art installation in her research studio in the Polish Hill neighborhood of Pittsburgh.

Her development sprint is focused on the debut of Manus, which connects 10 industrial arms with a single robot brain to illustrate new frontiers in human-robot interaction.

She’s been racing the clock to develop the software and interaction design to bring these robotic arms to life in time for today’s World Economic Forum exhibit, in Tianjin, China. Putting in 80-hour weeks in her warehouse, she’s faced the difficult task of taking two bots on loan from robotics company ABB and using them to simulate the interactions of all 10 robots that will be at the show.

Gannon relied heavily on her simulations to create the interactions for spectators. And she wouldn’t know for certain whether it actually works until she got the 10 machines onsite in China up and running.

The challenge of simulating robots in operation has traditionally driven roboticists to take on custom programming — not to mention big gambles and anxiety — because software until recently hasn’t reliably worked. 

Yet it remains a key issue for the industry, as logistics operations and warehouses shift gears to embrace robots featuring increasing levels of autonomy to work alongside humans.

“As we transition from robotic automation to robotic autonomy, art installations like Manus provide people an opportunity to experience firsthand how humans and autonomous machines might coexist in the future,” she says.

To be sure, Gannon’s gruelling effort in getting this demonstration off the ground underscores the industry’s nascent state of affairs for developing robotics at scale.

Robotics Help Arrives

Much of that is now changing. Earlier this year, we launched the Isaac Simulator for developing, testing and training autonomous machines in the virtual world. Last week, at GTC Japan, we announced the availability of the Jetson AGX Xavier devkit for developers to put to work on autonomous machine such as robots and drones.

Combined, this software and hardware will boost the robotics revolution by turbo-charging development cycles.

“Isaac is going to allow people to develop smart applications a heck of a lot faster,” Gannon said. “We’re really at a golden age for robotics right now.”

This isn’t Gannon’s first robotics rodeo. Last year, while a Ph.D. candidate at Carnegie Mellon University, she developed an interactive industrial robot arm that was put on display at the Design Museum in London.

That robot, Mimus, was a 2,600-pound giant designed to be curious about its surroundings. Enclosed in a viewing area, the robot used sensors embedded in the museum ceiling to see and come closer to or even follow spectators it found interesting.

Exhibiting Manus in Tianjin for the World Economic Forum marks her second, and significantly more complex, robotics installation, which required custom software to create interactions from scratch.

Bringing Manus to Life

Manus wasn’t easy to pull off. Once she arrived in China, Gannon had only 10 days with all 10 robots onsite before the opening of the interactive exhibit. Manus’s robots stand in a row atop a 9-meter base and are encased in plexiglass. Twelve depth sensors placed at the bottom of its base enable the interconnected robots to track and respond to the movements of visitors.

“There is a lot of vision processing in the project — that’s why its brain is using an NVIDIA GPU,” Gannon said.

This vision system enables Manus to move autonomously in response to the people around it: once Manus finds an interesting person, all 10 robot arms reorient as its robotic gaze follows them around.

To create the interaction design for Manus, Gannon needed to develop custom communication protocols and kinematic solvers for her robots as well as custom people-sensing, remote monitoring and human-robot interaction design software.

She says that up until now there haven’t been reliable technical resources for doing atypical things with intelligent robots. As a result, she’s had to reinvent the wheel each time she creates a new robotics piece.

The technical development for Mamus’s software stack took about two-thirds of the project timeline, leaving only one-third of the time to devote to heart of the project — the human-robot interaction design.

Future Robot Training

Using Jetson for vision and Isaac Sim for training robots could help developers turn those ratios around for future such projects. And they’re well-suited for development and simulation of industrial robots used by massive enterprises for warehouses and logistics operations.

Gannon’s mastery of training robots against such obstacles has garnered attention for her pioneering work, and she’s been called a “robot whisperer” or “robot tamer” for years.

She shrugs that off. “Now, with Isaac, I’m hopeful we won’t need robot whisperers anymore.”

Learn about availability of our Jetson AGX Xavier developer kit.

The post Robot Tamer Madeline Gannon: New Platform Will Bring Machines to Heel at Scale appeared first on The Official NVIDIA Blog.

In the Eye of the Storm: The Weather Channel Forecasts Hurricane Florence With Stunning Visuals

With Hurricane Florence threatening flash floods, The Weather Channel on Thursday broadcast its first-ever live simulation to convey the storm’s severity before it hit land.

The Atlanta-based television network has adopted graphics processing more common to video game makers in its productions. The result — see video below — is the stunning, immersive mixed reality visual to accompany meteorologists in live broadcasts.

Hurricane Florence slammed into the southeastern shore of North Carolina early Friday morning. Wind speeds of the category 1 hurricane have reached 90 miles per hour, and up to 40 inches of rain have been forecast to drench the region.

Warnings for life-threatening storm surge flooding have been in effect along the North Carolina coast.

The Weather Channel in 2016 began working with this immersive mixed reality to better display the severity of conditions with graphically intense simulations using high performance computing. This type of immersive mixed reality  for broadcast news has only recently become a technique used to convey the severity of life-threatening weather conditions.

In June, The Weather Channel began releasing immersive mixed reality for live broadcasts, tapping The Future Group along with its own teams of meteorologists and designers. Their objective was to deliver new ways to convey the weather severity, said Michael Potts, vice president of design at The Weather Channel.

“Our larger vision is to evolve and transform how The Weather Channel puts on its presentation, to leverage this immersive technology,” he added.

The Weather Channel takes the traditional green-screen setting — the background setup for visual — and places the meteorologist in the center for a live broadcast. The weather simulation displays the forecast via green screen, which wraps around the broadcaster with real-time visuals in synch with the broadcast. “It’s a tremendous amount of real-time processing, enabled by NVIDIA GPUs,” said Potts.

It’s science-based. The Weather Channel takes wind speed, direction, rainfall and countless other meteorological data points fed into the 3D renderings to provide accurate visualizations.

Video game-like production was made possible through The Weather Channel’s partnership with Oslo, Norway-based The Future Group, a mixed reality company with U.S. offices. The Future Group’s Frontier graphics platform, based on the Epic Games Unreal Engine 4 gaming engine, was enlisted to deliver photorealistic immersive mixed reality backdrops.

“The NVIDIA GPUs are allowing us to really push the boundaries. We’re rendering 4.7 million polygons in real time,” said Lawrence Jones, executive vice president of the Americas at The Future Group. “The pixels that are being drawn are actually changing lives.”

The post In the Eye of the Storm: The Weather Channel Forecasts Hurricane Florence With Stunning Visuals appeared first on The Official NVIDIA Blog.

Kiwi Delivery Robots Tackle Student Snack Attacks

What’s not to like about food delivered by a cheery little robot in about 30 minutes.

That’s the attraction of Kiwi Bot, a robot hatched by a Colombian team now in residence at the University of California, Berkeley’s Skydeck accelerator, which funds and mentors startups.

The startup has rolled out 50 of its robots — basically, computerized beer coolers with four wheels and one cute digital face — and delivered more than 12,000 meals. They’re often seen shuttling food on Cal’s and Stanford University’s campuses.

Kiwi Bot has been something of a sidewalk sensation and won the hearts of students early on with promotions such as its free Soylent  and Red Bull deliveries (check out the bot’s variety of eye expressions).

Kiwi Bot customers use the KiwiCampus app to select a restaurant and menu items. Food options range from fare at big chains such as Chipotle, McDonald’s, Subway and Jamba Juice to generous helpings of favorites from local restaurants. Kiwi texts customers on their order status and expected time of arrival. Customers receive the food with the app, and a hand gesture in front of Kiwi opens its hatch. Deliveries are available between 11 am and 8 pm.

Kiwi is partnered with restaurant food delivery startup Snackpass, delivering to its customers for the same $3.80 fee. For now, the delivery robots are only available around the UC Berkeley and Stanford campuses.

Reinventing Food Delivery Model

Kiwi is aimed at a unique human-and-robotics delivery opportunity. In Colombia, like in other parts of the world, it’s normal to get fast and cheap deliveries by bicycle service, said Kiwi co-founder and CEO Felipe Chavez. “Here it’s by car, and the the delivery fees are like $8. That gave me the curiosity to explore the unit economics of the delivery.”

Chavez and his team moved their startup — originally for food delivery by people — from Bogota to Berkeley in 2017 and applied to the Skydeck program.

The Kiwi team has ambitious plans. The company is working to develop a smooth connection between people, robots and restaurants, addressing the problem with three different bots. Its Kiwi Restaurant Bot is waist high — think R2-D2 in Star Wars — and has an opening at the top for restaurant employees to drop in orders. It then wheels out to the curb for loading.

At the sidewalk, a person unloads meals into a Kiwi Triike, an autonomous and rideable electric pedicab that stores up to four Kiwi Bots loaded with the meals for deliveries. The Kiwi Trike operator can then distribute the Kiwi Bots to make deliveries on sidewalks.

Packing Grub and Tech

Kiwi Bots are tech-laden little food delivery robots. They sport a friendly smile and digital eyes that can wink at people. Kiwi Bots have six Ultra HD cameras capable of 250 degrees of vision for object detection, packing NVIDIA Jetson TX2 AI processors to help interpret all the images of street and sidewalk action for navigation.

Jetson enables Kiwi to run its neural networks and work with its optical systems. “We have a neural network to make sure the robot is centered on the sidewalk and for obstacle avoidance. We can also use it for traffic lights. The GPU has allowed us to experiment,” Chavez said.

The Kiwi team underwent simulation for training its delivery robots. They also enabled object detection using MobileNets and autonomous driving with DriveNet architecture.  The company’s deep neural network relied on a convolutional neural network to classify events such as street crossing, wall crashes, falls, sidewalk driving and other common situations to improve navigation.

The Kiwi platform is designed for humans and robots working together. It’s intended to make it so that people can service more orders and do it in a more efficient way.

“It’s humans plus robots making it better,” Chavez said. “We are going to start operating in other cities of the Bay Area next.”

Lean more about artificial intelligence for robotics using NVIDIA Jetson.

The post Kiwi Delivery Robots Tackle Student Snack Attacks appeared first on The Official NVIDIA Blog.

What’s the Difference Between a CNN and an RNN?

The hit 1982 TV series Knight Rider, starring David Hasselhoff and a futuristic crime-fighting Pontiac Firebird, was prophetic. The self-driving, talking car also offers a Hollywood lesson in image and language recognition.

If scripted today, Hasselhoff’s AI car, dubbed KITT, would feature deep learning from convolutional neural networks and recurrent neural networks to see, hear and talk.

That’s because CNNs are the image crunchers now used by machines — the eyes — to identify objects. And RNNs are the mathematical engines — the ears and mouth — used to parse language patterns.

Fast-forward from the ‘80s, and CNNs are today’s eyes of autonomous vehicles, oil exploration and fusion energy research. They can help spot diseases faster in medical imaging and save lives.

Today the “Hoff” — like billions of others — benefits, even if unwittingly, from CNNs to post photos of friends on Facebook, enjoying its auto-tagging feature for names, adding to his social lubrication.

So strip the CNN from his Firebird and it no longer has the computerized eyes to drive itself, becoming just another action prop without sizzle.

And yank the RNN from Hasselhoff’s sleek, black, autonomous Firebird sidekick, and there goes the intelligent computerized voice that wryly pokes fun at his bachelorhood. Not to mention, toss out KITT’s command of French and Spanish.

Without a doubt, RNNs are revving up a voice-based computing revolution. They are the natural language processing brains that give ears and speech to Amazon’s Alexa, Google’s Assistant and Apple’s Siri. They lend clairvoyant-like magic to Google’s autocomplete feature that fills in lines of your search queries.

Moreover, CNNs and RNNs today make such a car more than just Tinseltown fantasy. Automakers are now fast at work on the KITT-like cars of tomorrow.

Today’s autonomous cars can get put through paces in simulation to test before even hitting the road. This allows developers to test and validate that the eyes of the vehicle are able to see at superhuman levels of perception.

AI-driven machines of all types are becoming powered with eyes and ears like ours, thanks to CNNs and RNNs. Much of these applications of AI are made possible by decades of advances in deep neural networks and strides in high performance computing from GPUs to process massive amounts of data.

Brief History of CNNs

How did we get here is often asked. Long before autonomous vehicles came along, the biological connections made between neurons of the human brain served as inspiration to researchers studying general artificial neural networks. Researchers of CNNs followed the same line of thinking.

A seminal moment for CNNs hit in 1998. That year Yann LeCun and co-authors Léon Bottou, Yoshua Bengio and Patrick Haffner published the influential paper Gradient-based Learning Applied to Document Recognition.

The paper describes how these learning algorithms can help classify patterns in handwritten letters with minimal preprocessing. The research into CNNs proved record-breaking accuracy at reading bank checks and has now been implemented widely for processing them commercially.

It fueled a surge of hope for the promise of AI. LeCun, the paper’s lead researcher, became a professor at New York University in 2003 and much later joined Facebook, in 2018, to become the social network’s chief AI scientist.

The next breakout moment was 2012. That’s when University of Toronto researchers Alex Krizhevsky, Ilya Sutskever and Geoffrey Hinton published the groundbreaking paper ImageNet Classification with Deep Convolutional Neural Networks.

The research advanced the state of object recognition. The trio trained a deep convolutional neural network to classify the 1.2 million images from the ImageNet Large Scale Visual Recognition Challenge contest, winning with a record-breaking reduction in error rate.

This sparked today’s modern AI boom.

CNNs Explained: Dog or Pony?

Here’s an example of image recognition’s role. We humans can see a Great Dane and know it’s big but that it is still a dog. Computers just see numbers. How do they know a Great Dane isn’t a pony? Well, that numerical representation of pixels can be processed through many layers of a CNN. Many Great Dane features can be identified this way to arrive at dog for an answer.

Now, let’s peek deeper under the hood of CNNs to understand what goes on at a more technical level.

CNNs are comprised of an input layer (such as an image represented by numbers for pixels), one or more hidden layers and an output layer.

These layers of math operations help computers define details of images little bits at a time in an effort to eventually — hopefully — identify specific objects or animals or whatever the aim. They often miss, however, especially early on in training.

Convolutional Layer:

In mathematics, a convolution is a grouping function. In CNNs, convolution happens between two matrices (rectangular arrays of numbers arranged in columns and rows) to form a third matrix as an output.

A CNN uses these convolutions in the convolutional layers to filter input data and find information.

The convolutional layer does most of the computational heavy lifting in a CNN. It acts as the mathematical filters that help computers find edges of images, dark and light areas, colors, and other details, such as height, width and depth.

There are usually many convolutional layer filters applied to an image.

  • Pooling layer: Pooling layers are often sandwiched between the convolutional layers. They’re used to reduce the size of the representations created by the convolutional layers as well as reduce the memory requirements, which allows for more convolutional layers.
  • Normalization layer: Normalization is a technique used to improve the performance and stability of neural networks. It acts to make more manageable the inputs of each layer by converting all inputs to a mean of zero and a variance of one. Think of this as regularizing the data.
  • Fully connected layers: Fully connected layers are used to connect every neuron in one layer to all the neurons in another layer.


For a more in-depth technical explanation, check out the CNN page of our developer site.

CNNs are ideally suited for computer vision, but feeding them enough data can make them useful in videos, speech, music and text as well.

They can enlist a giant sequence of filters — or neurons — in these hidden layers that all optimize toward efficiency in identifying an image. CNNs are called “feedforward” neural networks because information is fed from one layer to the next.

Alternatively, RNNs share much of the same architecture of traditional artificial neural networks and CNNs, except that they have memory that can serve as feedback loops. Like a human brain, particularly in conversations, more weight is given to recency of information to anticipate sentences.

This makes RNNs suited for predicting what comes next in a sequence of words. Also, RNNs can be fed sequences of data of varying length, while CNNs have fixed input data.

A Brief History of RNNs

Like the rising star of Hasselhoff, RNNs have been around since the 1980s. In 1982, John Hopfield invented the Hopfield network, an early RNN.

What’s known as long short-term memory (LSTM) networks, and used by RNNs, was invented by Sepp Hochreiter and Jürgen Schmidhuber in 1997. By about 2007, LSTMs made leaps in speech recognition.

In 2009, an RNN was winning pattern recognition contests for handwriting recognition. By 2014, China’s Baidu search engine beat the Switchboard Hub5’00 speech recognition standard, a new landmark.

RNNs Explained: What’s for Lunch?

An RNN is a neural network with an active data memory, known as the LSTM, that can be applied to a sequence of data to help guess what comes next.

With RNNs, the outputs of some layers are fed back into the inputs of a previous layer, creating a feedback loop.

Here’s a classic example of a simple RNN. It’s for keeping track of which day the main dishes are served in your cafeteria, which let’s say has a rigid schedule of the same dish running on the same day each week. Let’s imagine it looks like this: burgers on Mondays, tacos on Tuesdays, pizza on Wednesdays, sushi on Thursdays and pasta on Fridays.

With an RNN, if the output “sushi” is fed back into the network to determine Friday’s dish, then the RNN will know that the next main dish in the sequence is pasta (because it has learned there is an order and Thursday’s dish just happened, so Friday’s dish comes next).

Another example is the sentence: I just ran 10 miles and need a drink of ______. A human could figure how to fill in the blank based on past experience. Thanks to the memory capabilities of RNNs, it’s possible to anticipate what comes next because it may have enough trained memory of similar such sentences that end with “water” to complete the answer.

RNN applications extend beyond natural language processing and speech recognition. They’re used in language translation, stock predictions and algorithmic trading as well.

Also used, neural turing machines (NTMs) are RNNs that have access to external memory.

Last, what’s known as bidirectional RNNs take an input vector and train it on two RNNs. One of the them gets trained on the regular RNN input sequence while the other on a reversed sequence. Outputs from both RNNs are next concatenated, or combined

All told, CNNs and RNNs have made apps, the web and the world of machines far more capable with sight and speech. Without these two AI workhorses, our machines would be boring.

Amazon’s Alexa, for one, is teaching us how to talk to our kitchen Echo “radio” devices, begging all kinds of new queries in chatting with its quirky AI.

And autonomous vehicles will soon be right around the corner, promising a starring role in our lives.

For a more technical deep dive on RNNs, check out our developers site. To learn more about deep learning, visit our NVIDIA Deep Learning Institute for the latest information on classes.

The post What’s the Difference Between a CNN and an RNN? appeared first on The Official NVIDIA Blog.