New Look Ahead and HMI Technologies pave the Way for Truck Automated Driving

1. Introduction

The prospect of self driving 40 ton trucks may seem daunting for some. A machine in controland not the driver even for specific or dedicated road sections is a huge step for the industry.Can drivers trust this technology? Do the benefits outweigh the costs? These are questionswhich will no doubt be addressed in the next years.For others, it is the new horizon which has huge potential for the trucks and transportationindustries. At the same time automated driving has the opportunity to address specificconcerns of authorities, society and various industries. These include:

  • How can the vehicle support less experienced drivers in achieving best possible fueleconomy?
  • How can the job of truck driver be made more attractive?
  • How can accidents be further reduced?
  • Is automated driving time seen as rest time or full driving time?

The nature of the questions exposes a fascinating aspect of truck automated driving – thebenefit can be measured. It is well known that the transportation industry is ROI driven. Thesame is valid for the agriculture industry. It is no surprise that some of the first automatedvehicles are a type of commercial vehicles – tractors. This picture is a much clearer picturethan for passenger cars, where benefits for AD like improved comfort are very difficult toquantify and thereby to sell. Taking this as a starting point, it could be argued thatcommercial vehicles are the ideal next candidate for automated driving.

Continental is dedicated to making this possible and thereby supporting the manufacturers in their quest to making this happen. The role of Continental will be that of a component and/orsystem supplier. This will include the necessary:

  • radar, camera and laser sensors as well as the fusion thereof to an environmentalmodel to provide the vehicle with a 360° view
  • eHorizon system to provide the vehicle with long range look ahead capability to beable to prepare for forthcoming road conditions
  • holistic HMI concept and HMI systems which provide the required driver interface as well as conveying a feeling of trust to the driver

This presentation will focus on the latter 2 technologies and provide an overview of recenttechnology advances in these fields. It will also highlight key topics which still need to beaddressed in order to harvest the benefits of truck automated driving.

2. Potential Benefits

Measureable benefits are the key decision criteria for any new technology for the commercialtransportation industry. Automated driving will pave the way for new measurable benefitsmaking transportation both safer and more efficient. The benefit will be dependent on thelevel of automation. The SAE (Society of Automotive Engineers) levels can be seen in Figure1.

Level 1 already offers benefits through functions such as adaptive cruise control (ACC), advanced emergency brake assist (AEBA) and lane departure warning (LDW), which significantly improve vehicle safety. Typical level 2 functions like traffic jam assistant are also very useful to reduce the driver workload. These functions are already in or will find their way into the market.

But it is Level 3 and above automation which really opens the door for cost saving. When drivers no longer need to constantly monitor the vehicle trajectory, they can perform other tasks while the vehicle is driving. This can be very valuable for the fleet, since the driver can perform office task while on the road. These can include route and dispatch planning, expense reports etc. These tasks take valuable driving time either in breaks or back in the office. In addition to this the driver can safely work on his emails and other means of communication. From a driver’s perspective the job behind the wheel becomes more variable, less monotonous and therefore more attractive.

Driving times are strictly regulated today for good reasons. Drivers need to be protected. If the vehicle is in control and not the driver, this gives the driver the opportunity to rest if desired. The question is whether this should be seen full value driving time, rest time or something in between. If the answer is anything other than full value driving time then significant saving would be possible. At the moment this is only a theoretical exercise. It does however demonstrate the potential which automated driving could bring.

Automated driving opens the door for platooning, a very exciting use case, which is specific for commercial vehicles. In a platoon trucks can travel very close behind each over with a separation as low as 0.3 seconds. Obviously the following trucks should be automated. The advantage of this configuration is the slip steaming of the following truck(s). Tests performed in EU funded projects like SATRE [1] and more recently COMPANION [2] have demonstrated fuel savings between 5 and 15% depending on the vehicle separation. It is also interesting to observe that the lead vehicle will also see fuel savings of up to 5% due to the aerodynamic advantages of having a vehicle close behind.

Summarizing, automated driving has the potential to reduce operating cost for fleets by double digit percentages whilst potentially increasing productivity. This is motivation for the truck manufacturers and their tier 1 partners to drive technology.

3. Paving the way for safe and comfortable automated driving: The Dynamic eHorizon

Sensors which assess the surroundings of the vehicle are essential for automated driving. The most commons sensors are long and short range radar as well as mono and stereo cameras. These are already used in driver assistance systems like ACC, LDW and blind spot detection (BSD). LIDAR sensors are also available. These are frequently used in City Safe systems. The fusion of multiple sensors leads to a 360° model of the immediate vicinity of the vehicle. This is sufficient for automated driving up to a speed which is related from the range of the sensors. The range of the sensors is limited by physics. They cannot look around a corner or more generally further than the line of sight. This would be a limitation if there was not a solution called eHorizon. eHorizon provides vehicles with information on the road section for the next 3-5km regardless of the road geometry. Fundamentally the unit determines the position and the probable trajectory/path of the vehicle. The characteristics of the probable road segment are broadcasted onto the CAN as with sensor data. Each ECU attached to CAN can use this information to optimize its performance. Continental was first to market with this technology when providing it to Scania in 2011. eHorizon provides static road slope data in this solution. This static data is the first step in the eHorizon roadmap.

The next step is the connected eHorizon which today is close to serial production. Data recorded by the in-vehicle sensors such as slope data or traffic sign information will be used to enrich the ADAS map. Via an over-the-air connection this data can be uploaded to a backend server where it is merged into one common database together with data from other vehicles and subsequently shared with all connected vehicles. Once locally available the data will be integrated into the eHorizon road preview. This crowd sourcing approach allows the scope of the digital map to be extended. In markets with limited coverage regarding ADAS map attributes this will enable the introduction of eHorizon based systems. In more mature markets it will lead to an increased actuality of the map data.

Dynamic eHorizon is the latest development on the eHorizon roadmap. Now the digital map will turn into a high-precision and constantly up-to-date information carrier.The core of the system is the server based HD ADAS map. Mainly driven by Nokia HERE several map suppliers have started a new digitalization wave aiming at a lane level precise localization of road segments and map attributes.

The map database will constantly be updated taking into account dynamic events such as weather, accidents, or traffic conditions. The integration of information from a wide range of sources – from professional providers as well as from vehicle sensors – will make this a highly accurate self-learning system. The vehicle itself receives updates of the locally stored map as well as dynamic data relevant for the current position while providing sensor data as input to the map at the same time.

The vehicle will strongly benefit from the increased map precision and the integration of dynamic data. The applications of the eHorizon will extend significantly covering the whole range of today’s focus topics from fuel economy to connected powertrain and of course automated driving.

By adding information about the current traffic light status available from traffic management centers the dynamic eHorizon enables the implementation of a green wave assistant. The precise and up-to-date position of road constructions, accidents, hidden queues or traffic jam tails will significantly improve ADAS systems such as the emergency braking assist. For automated vehicles the dynamic eHorizon is an indispensible sensor enabling the application of predictive strategies that go beyond the line of sight or the range of the radar sensors respectively. For example, whenever the dynamic eHorizon identifies a tail of a traffic jam after a bend, an automated vehicle could gently reduce its speed, rather than initiating harsh braking once the vehicle sensors have detected the obstacle. Very importantly the eHorizon system can provide advanced notice when a handover from automated driving to manual driving is imminent or required and thus can trigger a type of takeover countdown.

Prototypes of this technology have been developed by Continental and are now under test. First tests results are promising with very short turnaround times for the data upload, aggregation and subsequent map and incident information download.

4. Future Truck HMI

Looking back the driver interface of trucks’ has remained essentially constant for the last years. It comprises an instrument cluster and multiple switches either in the steering wheel or on the dashboard. These components certainly have evolved over time. The older analog instrument clusters and gauges are making way for an ever increasing amount of digital information in integrated or separate displays. This type of HMI is well suited for SAE level 0- 2 automation in which the driver is in control of the vehicle at all times. The aforementioned represents one extreme. The opposite extreme is to say that the vehicle has total control of driving in all situations, i.e SAE level 5 automation. It could be argued that almost no HMI is required here. The Google Self Driving Car is evidence of this, although this is still some time before fully automated driving will be rolled out in higher volume.

We believe SAE level 3 will be the next step in the automated driving roadmap. The benefits for developing this have been clearly stated above. The major step forward here is the need for handovers to and from automated driving mode. When the vehicle is comfortable with AD mode it will offer and run this. If not it will demand that the driver resumes control.

Handing over control of a moving vehicle clearly involves trust. Transparency and/or information are probably the best trust builders. This is the classical role of the HMI. Thus the HMI will be responsible for conveying information which builds this trust in vehicle control of the driving situation. This becomes increasingly important in the wake of ongoing change in driver demographics with an ever increasing amount of inexperienced drivers expected in the next years [3]

All this places new requirements on the HMI. This will lead to new components and new functionality in the cockpit. The Continental vision of the cockpit of the future/2025 is shown in the figure below.

While the picture certainly contains well known elements like the instrument cluster and
steering wheel, new components can also be seen. The focus here will be on the new
components and their purpose in automated driving. They will be explained while running
through the following new use cases:

(1) hand over control to the vehicle, (2) automated driving and (3) returning control to the

(1) Hand over control to the vehicle
The vehicle can assess when automated driving is possible with the information from the
onboard radar, camera and LIDAR sensors together with dynamic eHorizon and navigation.
This possibility/recommendation is indicated in the new combiner head up display and/or fully
digital cluster as well as green flashing of a new dedicated AD (Automated Driving) hard key
in the steering wheel. The driver has the option of activating the automated driving mode by
pressing the AD hard key for at least one second. With this, the vehicle assumes control of
the trajectory.

(2) Automated driving
To build trust in the system, the vehicle should be able to visualize that it has correctly
identified the objects around and has these under surveillance. The augmented reality in the
head up display is ideal for this. It has the capability of marking objects which the vehicle
sensors have detected, e.g. a preceding truck as in shown in figure 4.

The Halo light strip around the inside waistline of the truck confirms this situation by shining
green in its front-facing section. The road sections where automated driving is possible are
shown in the full digital cluster in the form of a strip on the bottom edge. Thus the driver has
a constant indication how long this AD section is likely to last. Drivers now have the
possibility to use the CE/manufacturer/fleet specific tablet which has been safely clamped to
the dashboard until now. There is a possibility to clamp this device in a holder in the steering
wheel to enable drivers to work ergonomically in the driving position. Drivers can perform
other work while the vehicle is in control.

(3) Return vehicle control to the driver
Upon approaching the end of an AD section, drivers will be informed of the impending return
of control by a cascade of acoustic, visual and haptical information. 1 minute before
handover first visual and acoustic information will be given to the driver in the head up
display, via the speakers and in the Halo light band. This light band will perform something of
a visual countdown. The ambient lighting will change to prepare drivers for the forthcoming
handover. With approximately 20 seconds until handover, drivers will be asked to finish any
open work. The Halo light band has changed to illustrate the closeness of the handover. The
interior camera pointed towards the driver now starts to scan whether the driver concentrates
on the traffic ahead. Acoustic warnings become louder signaling the upcoming handover.
Vibrators in the seat could be the final element of the cascade. The handover is confirmed by
a long press of the AD hard key. In case of short term changes of the situation, that require a
faster take-over, accelerated alerting scenarios will be used.
In the unlikely event that the driver does not take control of the vehicle in time, the vehicle will
automatically initiate a safe stop maneuver. In this case the screen of the full digital cluster
and head up display are filled with the appropriate information. This safe stop scenario will
also be valid for sudden unexpected changes in the environment of the vehicle which force a
too rapid return of control for the driver to execute.
Prototypes of augmented reality head-up display, Halo light band, interior camera are
available in Continental test trucks and/or cars. The experience already gained with the test
systems will support the evolution towards automated driving.

5. Next Steps

With the aforementioned dynamic look ahead and HMI capabilities, major new components
will be available to support the evolution towards truck automated driving. Clearly further
steps will be necessary before AD trucks will be on the road. To give a quick overview, the
following section will briefly explain the most prominent challenges: legislation, vehicle date
recorders and testing strategy.

Today the Vienna Convention clearly states that the driver is always in control of the vehicle.
This is also reflected in national legislation and covers SAE levels 0-2. For SAE level 3 and
above adjustments are required in both. The more enticing questions is how AD time will
counted – as driving time or as rest time or something in between. Any changes here would
require adjustments in the tachograph legislation.

Liability/Vehicle Data Recorder
A further open issue is liability. Who is responsible in the unlikely case of an accident? Once
again the legislators need to define the necessary legal framework. Having said that, a black
box which records vehicle sensor data could at least support the post-accident clarification of
the accident cause. The advantage is clear. It has to be said that a black box is a sensitive
topic since it involves recording personal data. A solution with respect to confidentiality
certainly needs to be found for this.

Integration & Validation strategy (I&V)
It is the clear intention of all manufacturers that there are no liability issues. The best way to
prevent this is to offer solutions which have been developed for and validated in almost all
situations. Although ideal, this is definitely not realistic as it would result in several 100Mio
testing kilometers and incredible costs when using conventional I&V methods and processes
alone. Thus new I&V strategies have to be developed which guarantee the necessary
validation coverage while reducing costs and risks. Promising strategies contain new
reference models as well as probabilistic and mathematical methods. The combination of
both will finally enable the release of SAE L3 automation or higher levels.
From integration and validation perspective it is logical to proceed with an evolutionary
approach by starting in a “homogeneous” environment e.g. highways, to gain experience
herein and after this to extend the environment to more complex scenarios like interurban
driving. Following this line of thought, I&V and progress in automated driving will be an
incremental process.

6. Conclusion

The benefits of automated driving for commercial vehicles are definitely very enticing. The
possibility to measure the ROI and increased productivity will drive the industry to push the
development of eHorizon, HMI and further technologies. The benefits like improved safety,
fuel efficiency and total cost of ownership for the fleets will also help to motivate the
legislative authorities to offer the required legal framework. At the end these benefits could
drive the market in such a way that commercial vehicles are first to market with automated
driving – a fascinating outlook!

Article author:

Dr.-Ing. Michael Jörg Ruf, Continental Automotive GmbH, Villingen-Schwenningen