Category:Engineering and Design

From EuroVR Knowledge Base

Brief overview of area

Within Engineering companies, the take-up of 3D modelling technology has introduced a major cultural change. Not too long ago, process engineering moved from the use of pen and paper to the use of CAD software for drawing flowsheet and PI diagrams. 3D modelling involves yet another way to model the plant.
Large scale dynamic process simulation is primarily being used in the industry for design of process and control concepts, for automation testing and for operator training. The benefits from dynamic simulation are widely acknowledged: the model demonstrates how the system will behave, and how one should and should not operate the plant. In an ideal case, a dynamic simulation model should be in use already in the specification phase of a plant delivery project, at the time when operational practices are specified.
The problem in the take-up of simulation technology has been the poor connectivity of simulation tools to the other software involved in plant engineering. In order to have a functional simulation model, one needs to know the process component dimensions and the control parameters and to enter the data in the simulation tool together with the plant model. In principle, this information is available several months before the commissioning of the plant, but the data is scattered in different systems. Furthermore, the plant model may lack information concerning the plant topology, i.e. the order and connections of equipment, and the nominal state.

The major benefit of VR for Engineering & Design sector consists in replacing the existing physical mock-ups by interactive virtual prototypes. Automotive and aerospace manufacturers still widely use physical mock-ups, for example to validate assembly and maintenance operations (accessibility studies). Such mock-ups however do have numerous drawbacks. Essentially, they are expensive and they take time to build and modify. In the frame of faster time to market cycles, they do not always represent the latest version of the product and are often obsolete before they can be used. Manufacturers are therefore increasingly switching to digital mock-ups and Virtual Reality technologies that allow the realistic immersion of a human being in a virtual world, gaining substantial benefits in the process.
From these first VR applications in Engineering & Design, some feedback is available that points to a number of essential limitations as follows. VR software makes use of dedicated models, which are difficult to derive from the existing CAD models, interactions of the users with the virtual environments are often unnatural and that spoils the benefit of simulation with virtual prototypes. Concerning the dynamic behaviour of the interactive virtual entities it is still grossly inaccurate, restricting applications to contacts between rigid objects and to human manikins exclusively controlled in kinematical mode (ultimately, it is expected to use these virtual humans for human factors and safety analysis). Natural interaction between remote contributors is a problem while collaborative work by several users on a same virtual prototype is practically excluded.

At the moment, the major benefit of VR for Engineering & Design consists in replacing the existing physical mock-ups by interactive virtual prototypes. Automotive and aerospace manufacturers still widely use physical mock-ups, for example to validate assembly and maintenance operations (accessibility studies). Such mock-ups however do have numerous drawbacks. Essentially, they are expensive and they take time to built and modify. In the frame of faster time to market cycles, they do not always represent the latest version of the product and are often obsolete before they can be used. Manufacturers are therefore increasingly switching to digital mock-ups and Virtual Reality technologies that allow the realistic immersion of a human being in a virtual world, gaining substantial benefits in the process.

The larger aerospace and automotive manufacturing companies are now equipped, albeit generally still for test and evaluation, with "VR Centres" typically integrating a projection based visualisation system, sensors to capture the motions of a human operator and haptic interfaces. These "VR Centres" are currently used for:

• Validating design under development (external shapes, usability, assembly and maintenance operations, …)
• Collaborative work for design review and decision-making
• Production and management of the technical and marketing documents.

Virtual Prototyping (Photo CEA)

From these first VR applications in Engineering & Design, some feedback is available that points to a number of essential limitations:

• VR software makes use of dedicated models that are difficult to derive from the existing CAD models
• Interactions of the users with the virtual environments are often unnatural and that spoils the benefit of simulation with virtual prototypes
• The dynamic behaviour of the interactive virtual entities is still grossly inaccurate, restricting applications to contacts between rigid objects and to human manikins exclusively controlled in kinematical mode (ultimately, it is expected to use these virtual humans for human factors and safety analysis)
• Natural interactions between remote contributors is a problem
• Collaborative work by several users on a same virtual prototype is practically excluded.

To sum up, the main needs currently expressed by end-users are:

• Better CAD-VR integration
• VR for technical and marketing communication
• Virtual prototyping
• Collaborative work.

Current Engineering and Design projects

Vision and potential scenarios of use

It is worth to start this section by defining what Virtual Reality is. To avoid useless controversies, we may simply say that VR is immersion and interactivity. It derives that the main quality of a VR tool is to be transparent and/or of intuitive usage. In our Engineering & Design context, this may be exploited in two ways:

• Provide more intuitive VR-based man-machine interfaces to existing PLM tools
• Provide new functionalities fully relying on VR technology. In particular, the latter approach is desirable for:
  • Preliminary design where VR enhances creativity and allows fast testing of the devised solutions
  • Prototyping to experiment the system behaviour synchronised with human inputs and external events, especially simulating the interactions with the human operators (studies for accessibility, usability, safety …)
  • Communication displaying large and complex technical data in a most accessible manner
  • Training and in-situ assistance that are not strictly Engineering & Design, but that are also considered here as they share common resources and techniques.

On the other hand, Virtual Reality is not going to replace regular CAD and simulation software in a foreseeable future as interactivity is generally paid by computation approximations. It may thus be said that Virtual Reality is an extraordinary tool to implement natural interactions, to sketch a design and to point out its hard points. More accurate tools are often needed thereafter to refine and validate it in details.

The long-term vision derived from the above considerations may now be outlined through a number of key points:

• CAD-VR integration: Basically, models are transparently exchanged (from the end-user point of view) between the regular PLM tools and the VR modules.
• Natural interactions based on:
  • Intuitive interfaces (stereoscopic display, haptic devices, possibly brain-computer interfaces …)
  • Multi-modalities (combining different human input-output channels)
  • Efficient metaphors (translating information in the most intuitive way according to the user background and the available interfaces)
• Interactive simulation: The handled virtual objects are driven by the laws of physics while interacting with humans and the external environment. Different applications may put the emphasis on different physical phenomena (classical mechanics, fluid mechanics, radiology, chemistry …).
• Augmented reality to provide the humans intervening in the real world with information generated in a virtual environment or to allow these humans to enrich/correct the content of the virtual environment
• Collaborative tool: The future Engineering & Design VR applications will enhance collaboration between humans belonging to different kinds of organisation (big companies/SMEs), with different expertise or located in different areas, in particular through the collaborative manipulation of virtual objects.
• Scalable systems: Collaborative VR implies that persons may access a virtual environment through various kinds of system ranging from CAVE to laptops or to the foreseen future mobile immersive VR systems.
• SME needs: Breaking off with the current situation, tomorrow Engineering & Design VR assists SMEs in expressing their specific needs and develops products/services dedicated to this emerging market.

Most of these points are illustrated with the three following scenarios.

Virtual prototyping involving simple deformable objects (rather short term scenario)

Our first scenario deals with planning how to set a cable harness on a car or an aircraft while it is assembled. At the end of the design stage, a production engineer (let us call him José) receives a digital model of the new vehicle and of the considered harness, as well as the description of where the cables must go through. Three main problems are encountered:

• To define the harness setting procedure composed of numerous actions that are interspersed in the whole assembly process
• To check that the cables are long enough to be well connected
• To properly balance the workload of the human fitters who will be in charge of these tasks.

Since it is outside the short term scope of VR development, the last point is tackled in the next scenario. Regarding the other two, it is conceptually easy to apply the virtual prototyping paradigm to this situation and the motivation to do so is amplified by the fact that a different option on a car generally implies a different cable harness.

José thus starts a virtual simulation of the harness setting procedure. Using a 3D mouse, he controls the position of one point (typically an end) of a cable or a bundle of cables, while the simulation computes the motion of the rest of the harness taking into account cable weight and flexibility, and contacts with the already assembled components. Simulating the cable behaviours allows José to specify a validated procedure through trials and errors. For this purpose, he particularly checks:

• that each connecting operation takes place at a time in the manufacturing process when sufficient space is available around the connecting point
• that the harness may be arranged between connecting operations so as to avoid any interference with the assembling of the other parts (for instance, are we confident that the harness will not "fall" inside the vehicle when the engine is being laid in?).

Because the cable harness is a fragile component that features some very rigid parts (cables arranged in bundles or protected by a hard sheath), José then moves from his usual workstation in order to validate some delicate operations with haptic feedback. With this new interaction modality, he feels the pulling and flexing forces, and may thus verify that it is possible for the fitters to pull on a cable to connect it without damage.

Ultimately, the VR tools employed in the planning phase are used a second time to train the fitters that are in charge of the assembly tasks, while an intuitive 3D animated description of each harness setting procedure is quickly available on computers, replacing the paper procedures that were used a couple of years before.

Virtual prototyping of a human at work (rather middle term scenario)

Several years later, production engineers in an automotive company rely on a similar approach to simulate the work of a fitter fixing the interior trim of cars on the production line. Basically, the interior trim is made of highly deformable material while the worker has to slip inside the chassis, find supports and postures, and perform the assembly tasks which require tools as well as enough free space.

Using a virtual human duplicating the capabilities of a real person, a relevant scenario is first defined, and then executed thanks to a "scenarisation" software module implementing a high level control of the virtual human. A rough simulation is thus performed, providing numerous data regarding the fitter's workload and performances, the difficulty of its tasks and possible safety or health problems. For some important tasks, a finer analysis may be carried out in the more accurate condition of a virtual human controlled as an avatar of a real fitter.

From the data generated by this human simulation study, the arrangement of the considered workstation is optimised and the long term fitness of the real workers significantly improved.

Preliminary design of consumer products (rather long term scenario)

Several designers (Lucy, Dimitri, Volkmar and Monika) are collaboratively preparing the first drafts of a new product geared to a large audience (typically a mobile computer/telephone) with the help of a group of potential end-users supervised by Caroline. Many of them are in different locations because they are working at home and/or within different facilities. The whole drafting process is split into 4 steps:

• Sketching
• Part design
• Styling
• Users' validation.

Lucy is using simple drawing tools (black and colour pencils, white and transparent papers … on a classical drawing table) while her graphic production is captured by a video camera system. Dimitri is collaborating with her via a standard PC tablet. Their respective graphic production is partially analysed:

• To determine the type of drawing representation (plan, elevation, perspective views)
• To reconstruct some parametric shapes, as possible guides for next designing tasks
• To recover colours and lighting information for esthetical needs on the future product.

Furthermore, Lucy and Dimitri benefit from Augmented Reality (AR) devices, which provide visual feedbacks on the results of this analysis process, to help them correct the numerical draft of their work, and to introduce additional information on the objects they are designing (geometric references such as axis of symmetry, cinematic information provided by other designers which are collaborating with them, text annotations and so on).

On his side, Volkmar works with a CAD system for editing a numerical model of this manual draft, partly from video images of the manual draft that his colleagues have provided, partly from the structured information already found by the analysis procedures previously described. Volkmar is an expert in part design, and one of his tasks is to dialog with Lucy and Dimitri, to remove any incompleteness or inconsistency that blocks the analysis process, or to eliminate the possible topological and physical mistakes delivered by this automatic system. He wears a lightweight Helmet Mounted Display (HMD) providing an immersive access to the drafted product and he utilises his hands to refine moulding shapes or to select and apply virtual machining tools.

Using these CAD models, Monika’s team is investigating colours and esthetical issues. Some tangible prototypes are produced by a stereo-lithography system based on the numerical drafts. They feature some neutral aspects, because the focus of this designing team is to define colours and texturing properties and to build a set of skins for the future product. Once again some AR devices are used, but this time for simulating colours and texturing on the neutral tangible prototypes. In addition, the latter are haptically augmented in such a way that haptic texturing investigations can be done by the designers of Monika’s team via some tactile interfaces, to define and choose tactile features of materials which will have to compose the future object.

Last but not least, the visual and haptic AR interfaces used by the designers are also available to study the users' acceptance of the future products. This evaluation procedure is managed by Caroline over a set of potential end-users. Several tangible prototypes with specific geometrical shapes are ergonomically tested, while colour and haptic features found during the previous collaborative design process are evaluated. This time, the goal is to collect the quality preferences of the potential clients on these different possible products. The feedbacks of such end-users help Lucy, Dimitri, Volkmar, and Monika to choose the most popular shapes and aesthetics before refining the design.

Useful links