Intelligent Vehicle Initiative Vehicles

A final report for the National ITS Implementation Center

Authors:

Thomas A. Dingus

Andrew Peterson

Andrew Alden

Executive Summary

VTTI’s Center for Technology Development (CTD) develops, manufactures, implements, and maintains innovative data acquisition, collection, logistics, and analysis systems in support of transportation research. CTD continues to develop innovative solutions in response to the ever more rigorous requirements of VTTI’s research centers and sponsors. These developments include, but are not limited to:

• The technical capabilities and reliability of VTTI’s current DAS have been updated to provide for increased data acquisition rates and throughput via updating of communication and processing hardware;

• Creation of CTD’s dedicated Data Collection group which is tasked with retrieving data from vehicle- and infrastructure-based DASs and performing field repairs and initial data quality checks.

• Continued development of VTTI’s new highly integrated DAS that will significantly decrease installation times, increase data sampling rates and throughput, provide corrected vehicle dynamics data, and render improved video compression and quality,

• Continued development of machine vision capabilities related to driver, vehicle, and roadway metrics.

• Increased data storage and analysis capabilities. This includes implementation of a bank of dedicated, high-speed data export and analysis computers within the Data Center.

Introduction

For many years, VTTI’s CTD has designed and refined data acquisition technologies for large-scale studies. The Advanced Development, Data Acquisition, and Mechanical Groups within VTTI’s CTD design, develop, and build the data acquisition hardware, electronics, and software required to reliably and securely collect research data required by research staff. These state-of-the-art vehicle- and infrastructure-based data acquisition systems (DASs) are developed, built, and tested at VTTI’s facility in Blacksburg, VA. The current data acquisition system (DAS) comprises three general groups of measures: (i) DAS measures, (ii) vehicle network measures, and (iii) add-on measures. The general design characteristics for the DAS include the following:

Compatible with the vehicle (e.g., power obtained from vehicle battery, data from in-vehicle network).
Unobtrusive and non-invasive:

o Not distracting.

o Does not limit driver visibility.

o No permanent modifications to the vehicle.

o Minimal space requirement (e.g., for data storage unit).

o Automatic start-up, shut-down, and continuous operation.

o No participant intervention required for operation or data downloading.

Reliable performance in the often harsh operational environment of driving. Minimal data loss and automatic detection of failures.
Continuous multi-camera video recording system at 30 frames-per-second to capture driver’s face, over-the-shoulder, wide-angle rearward, and forward scene.
Ruggedness and crash survivability.

The DAS distributed data acquisition method provides a very flexible and maintainable hardware data collection system. VTTI’s DAS can be unobtrusively installed in vehicles to facilitate naturalistic driving behavior during controlled driving on the Virginia Smart Road testbed or in naturalistic on-road settings. The DAS instrumentation is concealed from the driver as much as possible. For example, cameras are mounted behind mirrors, while wires and other data recording equipment are hidden under interior panels. The DAS is designed to facilitate a rapid installation/de-installation and great care is taken to leave the vehicle in its original condition. Also, the hardware and software design team is mindful to minimize volunteer inconvenience and to reduce in-vehicle space requirements. As such, the main system box has been designed to be small and mounted under the rear package shelf within the vehicle’s trunk (Figures 1 and 2). The vehicle network box is located under the front dashboard. The incident box that allows participants to note incidents is mounted above the rearview mirror. Wiring is run though the normal wire chases on a vehicle to all the various network nodes, as well as to the cameras (Figure 3). All of the printed-circuit boards, including the firmware and data collection software, were developed at VTTI.

P1010012

Figure 1. The main DAS unit mounted under the “package shelf” of the trunk.

system2

Figure 2. The 100-Car DAS.

Figure 3. The mounting for the forward view camera behind the rear-view mirror.

VTTI’s DAS utilizes technologies such as high precision differential global positioning system (DGPS) and Doppler-based VORAD for position and ranging measurement. Vehicle-based data collection systems record multi-channel MPEG-4 compressed video/audio on PC-104 format on-board computers using either Microsoft Windows or Linux operating systems. Color, black-and-white and infrared video records external views (e.g., forward and rearward) and internal views (e.g., over-the-shoulder capturing vehicle instrumentation panel and driver hand position, face with eye glance, and pedal area). Other non-video data collected can include items such as: turn signal use, illumination, position/distance, speed, lateral and longitudinal g-forces, and yaw rate. The DAS interfaces with the Original Equipment Manufacturer’s (OEM) on-board vehicle diagnostic system to collect data such as throttle position, air bag deployment, and brake status. The system has been designed to add other sensors and data items as required for a particular project.

Ongoing development of VTTI’s next generation DAS will use advanced technologies such as digital signal processors (DSP) and field-programmable gate arrays (FPGA) and will feature broad-band wireless communication features to allow software updates, system health checks, and remote data collection all within a package that is about the size of a VHS videotape.

DAS Features

Global Positioning System

A GPS device is included in the DAS and is used primarily for tracking the instrumented vehicles and placing them in time and space. Data output includes measures of latitude, longitude, altitude, horizontal and vertical velocity, heading, and status/strength of satellite acquisition.

Lane Tracker

A VTTI-developed lane tracker, called the “Road Scout,” is included in the DAS. The Road Scout consists of a single analog black and white camera, a PC with a frame grabber card, and an interface-to-vehicle car network for obtaining ground speed (Figure 4). Note that the “grabbed” video frames are not stored but, instead, are processed algorithmically in real time to calculate the vehicle position relative to road lane markings.

Figure 4. Overview of Road Scout.

Road Scout has two interfaces for communication: a DLL version with exposed functionality and a version using a serial communication protocol. Road Scout can be configured to operate at 10 Hz on a 266 MHz Pentium PC or up to 30 Hz on an 800 MHz (or better) PC. The following variables are reported by Road Scout:

Distance from center of car to left and right lane markings (estimated max error < 6 inches and average error < 2 inches).
Angular offset between car centerline and road centerline (estimated max error < 1°).
Approximate road curvature.
Confidence in reported values for each marking found.
Marking characteristics, such as dashed line vs. solid line and double line vs. single line.
Status information, such as in lane or solid line crossed.

Once installed, Road Scout’s software automatically calibrates to determine camera position and thus no elaborate calibration procedure is required.

Yaw Rate

A yaw rate (gyro) sensor is included in the DAS and provides a measure of steering instability (i.e., jerky steering movements).

X/Y Accelerometer

Accelerometers installed in the vehicle are used to measure longitudinal (x) and lateral (y) accelerations.

RF Sensor

A radio frequency (RF) sensor can be included in the DAS. The RF sensor detects when the driver uses a wireless device. For example, if the driver uses a wireless phone, a flag is inserted into the data set to allow for quick identification of the event.

Vehicle Network

The measures that can be accessed from a particular vehicle depend on the make, model, and year of the vehicle. As such, it is possible that certain measures are only available for certain instrumented vehicles. The available measures are defined in a header file in each data set. The portion of the dataset that includes the vehicle network data typically contains measures of the following:

· Vehicle speed

· Distance since vehicle start-up

· Ignition signal

· Throttle position

· Brake pressure

Outside of the available vehicle network measures, other driver input measures that can be collected with sensors include the following:

· Right and left-turn signal

· Headlights on/off

· Brake pressure (if not available from the network).

Incident Box, Pushbutton, and Microphone

During vehicle orientation, the driver is instructed to push a red button on the Incident Box (Figure 5) whenever the driver is involved in a critical incident, near crash, or crash. This button serves to open an audio channel for 30 seconds. In this time, the driver can provide a verbal report of what occurred. A microphone is installed in the Incident Box to record the verbal utterances of the driver when the Incident Pushbutton is activated.

Figure 5. Incident box can be easily accessed by the driver.

Video Cameras

Digital video cameras are used to record continuous video of the driver and the driving environment. Currently, four video cameras are typically used; these are multiplexed into a single image (Figure 6). Typically, these views are: 1) forward, 2) driver's face camera, 3) over-the-shoulder, and 4) wide-angle rearward. Figure 7 shows the angle and approximate field of view for each of the four typical video cameras used in the video recording system. The forward and rearward camera views provide good coverage of the driving environment. The face view provides coverage of the driver’s face and allows researchers to conduct eyeglance analysis. The over-the-shoulder view allows visibility of the vehicle’s instrument panel as well as the driver’s hand position. A timestamp and data frame synchronization number are also included in the MPEG video file. The frame number is used to time-synchronize the video (in MPEG format) and the vehicle/performance data (in .dat format). Full time audio can be recorded and encoded into the MPEG video file. However, due to privacy concerns, this feature is typically disabled until the participant presses the Incident Pushbutton allowing them to record a 30-second verbal comment.

Figure 6. Locations and approximate fields-of-view of the four video cameras typically used.

video1

Figure 7. Split-screen presentation of five camera views used in the 100-Car Study. The lower right quadrant is more typically filled with a single wide-angle rearward view.

Onboard Data Storage

Data rates for the current data acquisition configuration are estimated at approximately four MB/min and are highly dependent upon desired video quality. This figure should be within ±10% of the actual data rate and is based on earlier and current VTTI data-acquisition-intensive projects. About 90% of the data typically come from four monochrome video cameras operating at 30 Hz using MPEG hardware compression. Calculations can be made to determine the frequency of data retrieval. For example, if data are collected every three months, the maximum storage required is as follows:

D_MAX = (100%+10%+20%)*(3 months*4.2 weeks/month*3 hrs/week*60 min/hr*4.0 MB/min) / 1000 MB/GB

D_MAX = 12 GB/3 months

Data Storage Capacity

How often the data from a DAS must be retrieved depends, in part, on the storage capacity of the system onboard the vehicle. In actual practice, several other factors play a role in deciding how often the driving data will be downloaded, but data storage capacity certainly dictates the maximum amount of time between data retrievals. VTTI’s current onboard DAS uses a 120 GB hard drive, which will handle data storage for approximately 200 driving hours. A ruggedized hard drive is used in order to survive on-road vibrations and potential mishaps or crashes. The systems (currently in use for the 34-Truck Study, the 8-Truck Study, and the 40 Teen Study) have been crash tested (35 mph, partial front-end crash at 30 g) and operated properly before, during, and after the crash.

System Booting/Powering

The DAS components (including the video) become active when the ignition system of the vehicle is initiated. The system remains active and gathers data as long as the engine is on and the vehicle is in motion. The system shuts down in an orderly manner when the ignition is turned off and pauses (on hold) if the vehicle ceases motion for a 10 minute period.

Data Acquisition Software

VTTI’s data acquisition software package is installed on all DAS boxes to collect and store the data brought in by the DAS in between downloads. This modular software is more reliable and easily configurable than previous versions.

Instrumentation Package (for installation in personal vehicles)

VTTI can install custom designed instrumentation systems into the participants’ personal vehicles if the study so requires. This method has proven very successful in recent studies (e.g., 100-Car Study and 40-Teen Study). VTTI’s mechanical group has designed instrumentation package housings that can be swiftly installed in volunteers’ vehicles leaving the vehicle in nearly its original condition. These temporary systems pose no crash hazard and can be readily reused in other vehicles. Use of VTTI systems is preferred to the use of commercially available data collection systems, for the reasons detailed below.

Some commercially available products provide camera views (front and passenger compartment), but the passenger compartment view shows the entire passenger compartment, including face views of all passengers. This configuration would not be acceptable to the Virginia Tech IRB or NIH’s Certificate of Confidentiality process, neither of which approves of capturing personal data about passengers without their permission. The cost of modifying these systems to meet IRB requirements would likely be prohibitive. These systems, even if modified, do not include comprehensive views of the drivers’ hand movements (the VTTI system provides an over-the-shoulder view), thus missing the opportunity to determine the exact nature of driver distractions involving a manual component. Finally, these systems, even if modified, do not easily allow integration with VTTI-developed software for eyeglance analysis.

Other commercially available systems do not include camera views, and thus miss the opportunity to capture driver eyeglance patterns and hand movements. When events are captured by the data sensors in this type of system, there is no visual record of factors that may have precipitated the event. Valuable detail on how drivers respond to particular driving situations is thus lost with these systems. For example, if a hard deceleration is captured, there is no way of knowing whether the deceleration was prompted by a lead vehicle braking, a traffic signal, or some other stimulus. The final drawback to all commercially available systems is that they lack a meaningful number of sensors. Thus, the number of triggers that can be searched for potential driving conflicts is limited. Some systems do not include GPS systems, which can be useful in post-hoc data analysis for determining geographic clusters of incidents which may indicate areas where participants encounter difficulty on a regular basis. Many systems include only such items as speed, longitudinal acceleration, and lateral acceleration. Finally, some systems (especially those without camera views) do not allow capture of such important measures as reaction time, eyes-off-road time, and avoidance maneuvers.

Data Collection

Data collected by the DAS are saved continuously throughout the data collection period. Periodically the data are retrieved from vehicles by CTD’s Data Collection Group (DCG). The DCG staff work with researchers to devise data retrieval strategies that ensure regular, reliable, and secure data collection while working with the constraints of participants’ schedules. The DCG staff performs multiple tasks when visiting a data retrieval location. The hard drive is removed from the DAS and initial data quality checks are performed on a laptop computer equipped with a drive reader. The current configuration of the DAS is verified and software/hardware upgrades are performed as required. The staff also makes minor repairs to the DAS in the field (such as replacement of hardware interface boxes, cameras, or radar antennae). Hardware and data collection status are then updated via a web interface and the hard drives are submitted to the Data Service Center staff along with a chain-of-custody record.

Data Services Center

Once data are collected from the field by the DCG, it is transferred to VTTI’s Data Services Center (DSC). The DSC is operated cooperatively by CTD’s Data Services Group and VTTI’s Information Technology Group (ITG). The Data Services Group performs the following functions within the Center:

· Uploading of collected data to VTTI’s dedicated, large-scale (VLS[SEL1] ) data storage area network (SAN[SEL2] )

· Management and large-scale storage of data

· Data quality control and assurance

· Data tracking

· Creation and maintenance of VTTI’s proprietary data viewing software

· Database building, optimization, and maintenance

The Information Technology Group performs the following functions within the DCG:

· Specification and acquisition of hardware and software

· Maintenance, repair, and updates of hardware

· Security and access control management

· Data backup and archiving

· Data reduction workstation configuration and maintenance

Dedicated Data Virtual Private Network

VTTI has quickly become an industry leader in naturalistic driving research, and this has created the need for advanced data services. The recently completed 100 Car Study alone produced over 6 TB of data (which included over 2,000,000 vehicle miles and 42,300 hours of driving). New capabilities added to the DAS and larger participant numbers have increased the volume of data collected during the data acquisition process.

Research data are stored on the VLS Data SAN that operates within a dedicated virtual private network (VPN). It is isolated from VTTI’s operational VPN and all other networks including the World Wide Web by a high-end firewall appliance managed by the ITG. All network connections within VPNs at VTTI are high-speed Gigabit Ethernet.

The Data Services Center facilities are located within a newly constructed building at VTTI and feature the following:

· Emergency power provided by an onsite diesel generator supplying backup power for the data center, emergency lighting and the telecommunications closet.

· External wide area network (WAN) speeds equal to an OC-3 (45 Mbps; approximately 30 times that of a T-1 connection).

· A dedicated climate control system with a backup contingency system.

· Remote monitoring alarm system for indication of fire, smoke, intrusion, power outage, climate control failure, hardware failure, water presence, high temperature, and other situations.

· Elevated flooring system for flood protection.

· High physical security with steel reinforced structural walls.

· Limited personnel accessibility.

· Two 600 SF secure data reduction laboratories housing high-end computer workstations.

VTTI’s Data Services Center server room houses the following equipment:

· 250+ Dell branded business class desktops, laptops and laboratory workstations

· 15+ high availability, high performance DELL PowerEdge servers (Figure 8)

· Over 60 Terabytes (Tb) of redundant high-speed storage with short-term expandability exceeding 100 Tb.

· Redundant optical SAN switch/routers.

· Large capacity backup system that includes a tape library capable of handling 12 Gigabytes of data per minute.

· All network connections are high-speed Gigabit Ethernet; VTTI was a pioneer in implementing this technology to every network portal.

· A dedicated IT staff with over 30 years of combined experience in the information technology field.

613474

Figure 8. The VTTI Data Center is environmentally controlled and physically secure.

Data Reduction

Data Reduction Laboratories

The mission of the VTTI data reduction laboratories is to process the raw data collected in VTTI studies and reduce it into a format that allows specific research questions to be accurately and effectively answered. Once study data have been uploaded to VTTI servers, Data Services facilitates researchers in viewing and maintaining their datasets. For this process, Data Services developed a data viewer in-house that supports the synchronizing of video and performance data.

Data Reduction Capabilities

Data Services uses two full time data reduction laboratories that will contain 32 data reduction stations when they are fully staffed (Figure 9). To maintain data security, all data reduction machines are connected only to the data server and maintain no outside connections. Additional data reduction laboratories can be developed and staffed as needed. The typical data reduction staff is 20 to 40 persons working in four hour shifts.

613470

Figure 9. VTTI’s data reduction laboratories.

Disclaimer