Sensor Data Visualization

At NIO, we strive to achieve autonomous driving at the highest certification level. To accomplish this, relevant on-road data has be collected by our fleet of vehicles equipped with various data sensors. Once trips are driven and data collected, our data scientists require an application to search for and to validate the data. 

Because of the many moving parts involved for search, such as correlation between different data types, vehicles, dates, etc., I've spent many UI component iterations on whiteboard with project management teams to achieve an agreeable holistic layout for search and search results. The final search design features clean aligned input boxes and  very scannable data views in the results area. 

Unlike search, wireframing for the data visualizer was a more straightforward process. The freedom to explore a broader range of layout ideas was restricted by a fundamental constraint - the video playback ratio has to stay at 4:3 in a page that's partially responsive, and we had to leverage what space is spared for data description. The resulting page satisfies our need for camera/LIDAR playback, multi-sensor display toggle, data occurrence timeline, and quick skip to specific data type. 

During offloading content to database, we quickly learned that lengthy camera videos could cause system and bandwidth slowdowns. Not only that, we discovered that operators who are indexing data types during the vehicle drive could cause a delay from which the object appears on video vs how fast the operator could insert that data in real time. We proceeded to implement UI measures to allow different bitrate playback of video to render smoother playback and UX flow to edit and adjust data/time stamps when needed. 

Working on this app was extremely joyful. The ability to present so many different data and media types while having so many moving parts has been a fun challenge. Over the years I've learned that even designers would have to think like engineers just so design could be implement realistically and faithfully. With this project I've come to realize that backend performance also plays a pivotal role in providing a smooth user experience.