Rail Digital Transformation
By Paul Williamson
February 9, 2022
0:00 / 41:31
Paul Williamson00:08

Hi, this is Paul Williamson from Inter Reality Labs. I'd like to take you through our strategy for digital transformation in the Rail industry. To start with we've been working in the Rail industry for anything up to the last 12 years in Virtual Reality, Mixed Reality and engineering, and we're partnered with major consulting companies here in the UK, so that we can deliver both operational consulting and services in the rail industry, as well as training.

What is Mixed Reality?
Paul Williamson00:37

So what is Mixed Reality? Mixed Reality is the combination of the Virtual data world superimposed directly onto the Physical World. By that we mean physically attached to our workplaces in a very precise manner, allowing us to create new working experiences and unlock a different value and different efficiencies in the industry.

We work across three main groupings. These are highly flexible. We group them in terms of Simulation and Guided which is aimed at more rapid transformation of workplace, workplace skills; Remote Inspection and Audit, which allows us to combine spatial and asset information with the inspection in audit workflows, and then Connected Field Service, which is the combination of collaboration on time people management, everything they need in order to provide a better operational service.

So we apply that in a number of areas. First of course, is Training. We also work in Project Asset Management, Operations Support, and Project Support. We work with some great clients and great partners and some really good people to explore how this very new technology can be applied into the rail industry in an increasing number of ways.

The Mixed Reality Technology Stack
Paul Williamson02:07

So this is a continuously developing and it's built upon what we call the Stack. So the Stack is a deliberately careful combination of Tool- grade Mixed Reality applications. So that's the row along the top, where we provide the guides Microsoft Dynamics, 365 Guides and Microsoft Dynamics, 365 Remote Assist tools, which together can deliver the majority of the value in mixed reality applications in business today. Combine that with the Power Platform that gives us the ability to write low or no code interfaces into operating systems, link into web and provide that extra level of interactiveness into real-time systems. That's essential for workers out there.

It's made possible by the Microsoft mesh platform which is that combination of Azure and Machine Learning, Collaboration, Gesture, Translation, Speech Translation, all there. These really, really hard problems have been solved and are available for us to reach into and use as part of the Microsoft Mesh network.

We deploy those on a range of devices. We're not locked into any particular device. However, we are a Microsoft partner and one of the reasons we're a Microsoft partner is because the HoloLens 2 is out now. The best Mixed Reality platform that's available at the moment by quite a long way, and so , that particular device enables us to achieve everything we want to achieve using these tool grade Mixed Reality programs, and also.

We can work with Virtual Reality, both on proprietary platforms and the Windows platforms, PCs, and mobile. So supporting those across all the different platforms as well. And we provide haptic equipment and immersive experiences from our Simulators. Especially in the rail industry. We've got many, many years of experience in building those and some unique capabilities there.

Railway Training Simulators
Paul Williamson04:05

So I'm going to first off take you through the next few slides. We'll talk about our specific railway Simulation capability. So you can skip down to the mixed reality section if you want using the menu to your left. But to follow, we're going to take you through what we do in simulators.

So Railway Simulation is quite a complex environment. Here we build really nice looking and very effective immersive simulators. This is a full cabin Siemens Vectron freight locomotive. It's also built on one of our proprietary three axis motion systems.

That system here is just being taken through its full range of motion, It's designed to give the feeling of motion on acceleration, deceleration, curves lumps and bumps with regard to shunting and create a more immersive experience.

The critical thing is we build these ourselves. We don't buy them from anyone so we can build three axis, five axis, a six axis systems, and low-profile to operate on existing corporate business floors -250 kilograms per meter squared. We are definitely unique in the marketplace in that capability.

So our full cabins can be full replicas using real railway hardware which is integrated using our IOT network into virtual worlds and then wrapped around using high resolution screens into a very, very immersive. We also deliver these as console, as you can see to the bottom left of the screen.

Console Simulators & Remote Training
Paul Williamson05:36

These consoles are important because all of our simulators can be operated by remote. So where you have an instructor in one location and a student in another location, maybe three to 500 miles away they are able to work together to run through an effective training simulation.

I'll run you through a short demo view on how this one works. So here you can see the Driver is running through a rescue scenario, and I have just, super-imposed the views of the Instructor in the bottom right hand side of that little window,

All of the radios, all of the safety equipment is real and it works using our software. So that's the UK GSMR radio system. Which is a hardware simulation.


"Hello, this is the driver on One-Sierra-Four-Five standing at Papa-November..."

Paul Williamson06:25

To the right, we can see one of our instructors working with HoloLens and stepping through just above them, the virtual training mode, where we deliver our simulators using a HoloLens so that new instructors can actually learn how to use the simulator without needing to go through formal training.

Mobile Simulators and Containers
Paul Williamson06:41

Systems can be mobile so we can install a standard or motion systems inside moving containers, which can be expanding or classic 20 or 40 foot containers as well. And we can provide the operation support to locate relocate, and also prepare the sites for these if neccessary.

Virtual Railways
Paul Williamson07:03

One area we're quite famous for in the Railway Simulation arena is the quality of our massive Virtual Railways.

These environments can contain a a thousand miles of track and we're able to deliver these in very high quality using one computer. So that allows us to deliver extremely realistic and sophisticated simulation with a minimum of hardware. Now, without all the support costs that come with having to run a network of six to eight computers to run a simulator.

Freight Simulation demo
Paul Williamson07:35

Gonna run you through a very quick overview of one of our freight simulations. This is current as of November, 2021. You can see the lighting effects and this is filmed using actual simulator code

The point is you can see the extent of the facility that this railway is based in.

This facility is an extensive steel manufacturing plant contains nearly 3000 kilometers of track.

And the shunting locomotives on 24, 7, moving heavy iron or slag and other products in and out of the furnaces all day.

there's no limit to the consists you can build using our locomotive builder, and we can provide a full shunting simulator as well.

In Railway Simulation we work under the SMART brand.

West Coast Mainline Simulator demo
Paul Williamson09:46

Here's another example of our extensive track networks, which also highlights the detail that goes into the different train simulations, including moving bogeys , pantographs and ancillary equipment. Not to mention our interiors.

Train Models and Virtual Reality
Paul Williamson12:32

So, two views upon the different sorts of track networks we've been producing over the last couple of years. And you'll notice that the trains that we modeled are extremely realistic both interior and exterior are modeled in as close a detail as we can.

One of the reasons for that is we want to be able to unlock the training position from just sitting in the driver's seat to actually be able to move around on the platforms, to be able to conduct procedures and also train other staff.

So we'll demo that with our VR version of the same simulation model. So exactly the same code, exactly the same graphics, but interfacing using Virtual Reality instead of a cabin or a driver's console.

So this is our PC interface. As mentioned before, it's using the exact operating code from a full-size simulator. So the investment that's been made in modeling all of these systems and how the train actually works can be reused directly in a Virtual Reality environment. Well, the software systems, alarms, train safety, radio work exactly as they do on the real train and work exactly as they do in the simulator. So that training content can be taken from one system to another.

You can even replay the same scenarios.

As we go, we can see the graphs of gradient, performance, driving energy, whatever the driver needs to see.

We can also have a range of events, faults, failures, and other features that need to be trained on the railway.

Freight Virtual Reality Inspection Training
Paul Williamson15:09

Second example of that is the same concept, but this time I'm going to use it for Freight Inspection.

So the same environment, again, using the PC or the VR interface allows us to walk up and down a consist. We can select and interact with railway features, both the infrastructure and on the rolling stock. We can expect bogeys and wheels for damage. We can run through inspection scenarios, score items that are missed or found, and also go through complex fault fixing and troubleshooting exercises, both again on the vehicles or on the infrastructure.

This is a realism and interactivity of these is limited only by the data we can get.

And the imaginations of the trainers

Detailed stations on the Moscow Metro demo
Paul Williamson16:15

My last example is a run through the Moscow Metro, which we'll show you some of the environmental graphics and just some of our experience in working in Metros

This environment's a good example of the detail gone into in both railway depots, surrounding vehicles and environments.

We have some traditional service tunnels, platforms. Some of the beautiful platforms that the Moscow Metro is famous for.

Foliage varies by seasons. So in Winter time it's covered in snow Autumn goes brown. Summer-time bright green. That's tuned to the environment of the country that we're working within.

Some areas, we have to go to a very high level of detail in texture and the contents of stations in here we have some of the beautiful stations from the Moscow Metro

Testing and Training with ETCS
Paul Williamson19:07

Back in the operating world. We also use our experience to provide testing and training services on the real equipment. So this is a good example of the work we did on ETCS testing and training for the implementation of ETCS on the Heathrow Express.

This had two elements to it. The first one was because of COVID we needed to provide some training support and some driving support for the training.

COVID rules meant we could only put one person in each cabin, which meant that the experienced instructor driver had to sit behind in the saloon. So in order to see exactly what was going on and also to be able to stop the train in time, if there was a problem.

We provided a range of high speed cameras and integrated views from the front of end of the train and the driver's cabin into the saloon using high-speed gaming screens. So that they could see in under six milliseconds exactly what was going on in high definition, coming through the front of the train and were able to, if they needed to, hit the emergency stop button, which had been installed on the desk.

And so this gave them the ability to combine inputs and views from the train equipment. So if we look at the bottom right-hand side, we can see the real time information, which has been streamed out of the train ETCS interface.

And then the third, we have a web interface, which shows the Network Rail routing. So we can see exactly where every train is in that particular area in real time, including this train.

And of course the highest (quality) views from both the cabin and the world outside. These were taken at night just to protect the privacy of the driver.

Operations testing using Mixed Reality
Paul Williamson20:59

And then a second view where we did testing with the HoloLens. So this technology allowed, in situations where we only had access to get one person on a train, to actually test the systems and the documentation by providing remote support to the driver who could operate hands-free

"that's fine... that's fine. Back to the documents.....".

So from a remote project office, somebody working from home or from the office, then the engineer on-site can interact with the train equipment, test documentation and keep projects going, even when they may be, may have been delayed without, because it wasn't possible to get more people onto the train.

"... today, 30 seconds, if it goes through..."

And we tested quite a range of Mixed Reality and Assisted Reality equipment for these applications.

".... is it running? Yes was just unable to do brake tests because we ended up Depot that we should be getting into ETCS selection level displayed...."

Remote Observation & Debrief
Paul Williamson22:10

So that allowed us to expand on remote observation and debrief capabilities. So not only are we linking directly to simulators or other training environments, we're able to bring in remote signals, we'll be able to bring in the viewpoint of people out in the field, on the train, on the track to provide interaction with classrooms and trainees who may be hundreds of miles away in a safe classroom.

We're also able to provide a level of assurance, so that trainees out in the field can be viewed by experts in the office, for assurance of safety and also for assessment of competence.

Mixed Reality Applications
Paul Williamson22:54

So that concludes the Railway Simulation part of things. What I'd like to do is take you through the key elements of Mixed Reality with the dynamics toolset. So. Dynamics 365 Mixed Reality is the first element in our solution stack for Mixed Reality. Our solution stack has four key pillars to Mixed Reality.

The first one is Remote Assistance. The second is Guided Workflow. The third is Collaboration and the fourth is Digital Twin. Now a simulation is a very, very good example of a digital twin application.

So what I'd like to do is take you through some general industry applications that we've found for Mixed Reality, using a curated set of examples, which get the point across better than anything else we've found.

Remote Assistance
Paul Williamson23:47

So to start with, I'm going to talk about remote assistance and introduce a really good example that has been built in to their business services by Hewlett Packard.


There's a motor failure on the main press. Okay. Nobody's just me. This job has to go out today or find me

I've never used.

Good morning. This is Sarah from HP. How can I help? Hi, I'm James. I'm getting an EBM motor failure. What do I do? It's not a problem. This is what I'm hearing. Let's take a look.

Unbelievable. That was actually a lot easier. Glad to hear it. Have a great day. The HP XR services, a new standard of customer support that is easy and immeidate.

Paul Williamson25:20

So, this is a really good example of a market ready technology. What we call a Tool- grade Mixed Reality technology that is immediately available to be baked into your existing services.

Use Cases for Mixed Reality
Paul Williamson25:32

We have a lot of clients who already sell these services as a new profit center in their business. They're using them in quite a wide range of areas. We just have a great customer service and repair scenario that could have been used as a training scenario as well. But we can extend this through training, through operation repairs, inspections, customer service, and we're seeing it applied in a whole range industries.

Extensively tested in Automotive, Energy and Manufacturing, and increasingly used in Healthcare, Construction and Retail, and as well as Aerospace, but also in Rail as well. We're pioneering this application into the Rail industry based from the UK, but supporting customers around the world.

Hololens 2 Mixed Reality Headset
Paul Williamson26:22

We're working with the HoloLens to is our primary Mixed Reality tool.

The reason for that is it gives us a completely hands-free interface, but also a very wide field of view, which is unrestricted and in which we can place and interact with our holographic Mixed Reality content.

HoloLens 2 itself comes in two different flavors. So we work with the HoloLens 2 and the Industrial Edition.

That's a system has been designed primarily for clean rooms and areas where a level of intrinsic safety is required. It is equivalent to ATEX level 2, and we provide what we call ARMR Field Kits.

Augmented Reality, Mixed Reality field kits, which provide a mobile tool-kit to convey both a device (and any device), a full days worth of power, network, Wifi, LTE cooling systems; compatible PPE, with adapters and proven systems that we've developed to actually make most PPE possible with a mixed reality headset; and also with managed Support and Help Desk if required.

Remote Assistance detailed demo
Paul Williamson27:32

This is a great example to walk-throigh the Remote Assist Concept.


"...everything is on the line. Employees are the heart of every business. It's their resourcefulness ingenuity and drive that, get the job done. But even the best employees need help from people who aren't there.

Now employees on site can work side by side with experts in remote locations, in an instant they can share what they see lend a helping hand walk through the process. Step-by-step helping solve problems in real time.

This is a new vision for work. Now, employees can work smarter and bring in the people they need with just a tap. They can troubleshoot repair and perform with confidence and speed. They can move from. For critical information and to view and stay focused on the task. They're hands free to do the work.

This is collaborative problem solving at full speed. Your people United across time and distance, more connected, more informed and more efficient. Now employees can work together from. They can go there without being there. This isn't the future. This is here and now.

Paul Williamson29:46

So, as you can see, that is a system which has supported across both the handheld and mobile devices. So as long as they have an AR camera or an AR capability, so anything reasonably recent will be supported in that. Businesses use Samsung which of course are one of the the more advanced AR supporters on Google. And of course, any of the Apple iPhones can also support this technology.

Rapid payback of Mixed Reality
Paul Williamson30:09

So being able to work both with an immersive headset or an AR headset, and also with a tablet device is allowing companies to get these benefits out into the workforce more quickly than you might think. The payback on these is really fast.

Just the simple cost of simple savings of avoiding downtime for major equipment, the frustration and lost sales through disappointed customers -if you cannot get their equipment up and running fast enough, and also huge reductions in travel costs.

You do not need to send your very high cost and very rare experienced employees out to site anywhere near as much , as you would, if you didn't have this technology this can cut travel time down to nearly nothing because one expert can support any number of interactions on remote sites, anywhere in the world in a given day, without having to move from their office, exceedingly powerful technology.

Mixed Reality Guided Workflow
Paul Williamson31:09

So that's Remote Assistance. I'm going to introduce the second pillar of remote technology, which is Guided Workflow. So Guided Workflow uses Mixed Reality, Mixed Reality technology to guide an employee through a Workflow - it's pretty simple.

So you'll see how the technology is making that possible in this very, very good explanatory video.


Work today is more complicated than ever. The old ways of training cannot get new employees up to speed as fast as the experienced people are retiring.

We need a better way forward.

Introducing Dynamics 365 Guides. Simple step-by-step instructions are easily written on a PC and then holographic parts and icons are simply picked up and placed on the machine where the work is happening.

From day one, employees are often expected to get work done, but the complexity can be overwhelming.

Now employees can learn by doing this is the tool that works with existing processes, allowing the manager to get the right Guides to their people.

The instructions move with the employees, pointing them to the tools and policy. Showing them exactly where they need to apply them.

Employees, get up to speed faster with fewer errors and more confidence. A simple glance moves them through the instructions, meaning their hands are free to do the work. They can learn at their own pace without having to rely on other workers around them. The overall experience is comfortable and in.

For the first time a manager can see detailed information about how the employees are working and how their skills are growing. This makes continuous improvement faster than ever before. All of this helps employees make the complex simple so they can learn and adapt faster and organisations can stay competitive..

About Guides and Customer Benefits
Paul Williamson33:39

So Guides, Guided Workflow. It looks a lot like PowerPoint and it's about as simple, the ability to break down a complex workflow into manageable chunks displayed in spatially linked cards. They can move with you, but they can also be completely linked to a particular piece of equipment or a particular location.

They allow us to standardize workflow and to test, to measure whether or not it's been complied with. Also to give someone the ability to follow a workflow straight off the bat.

So this has huge advantages of being able to reduce error and rework and speed up tasks. And you can see some of those elements, those boxes flooding around the hand, they apply some automation. So as soon as somebody moves into a particular area, then they can have useful information displayed to them.

So it can be quite sophisticated and interactive in its implementation and being able to reduce downtime, get someone up to speed and productive as quickly as possible and make sure that they follow the process correctly every time. We also use these for inspection scripts, so we can run somebody through an inspection to make sure that they're actually inspecting the necessary equipment on a wagon.

These can also be linked into real time operating systems so that we can interact with real-time information systems and Applications.

Supporting Rail Operations
Paul Williamson35:15

These tools enable us to support project Asset Management, Operations and Projects in some of the ways that you've seen. A really good example of this is what we call the Connected Project Office, where you may have someone working out in the field and they may be responding to, or interacting with data, which has been collected through the IOT cloud

. From the field they're able to bring in remote Specialists and also connect to their Project Office who can send them the data they need without them having to reach into or log into other systems . See exactly the documentation, the videos, the drawings, the data that they need to solve a problem on the spot.

The information can also be escalated through , the Project Office and into the Boardroom so that people decision-makers can see exactly what the situation is on the ground, can see what the people on the ground see and make the best decisions based upon the best information.

All of that can be linked through to the supply chain using automated data exchange and work orders in order to act on the decisions that have been made both in the field and in the project office. Solve problems a lot faster and chew days, weeks and months out of some project timeframes.

The Stack - Microsoft Mesh Platform
Paul Williamson36:32

Those the key elements to our solution stack in Mixed Reality. So we've run you through some examples of Virtual Reality, the HoloLens, PC and mobile applications and Simulators, and demonstrated what we do with Guides and Remote Assist to start with.

Now with Power Apps and most importantly, with Microsoft Mesh, this is a very new.

Microsoft Mesh we're not offering that today - it is an exciting development, which brings to the Mixed Reality field, all of the necessary sophisticated solutions, such as AI Machine Learning, Natural Language Processing , massive data base mining and other real-time services, which solve really, really hard problems, make them easy and then package them in ways that we can bring into our Mixed Reality applications.

So Mesh is a really good example of that. That's a concept and we're betting it's going to be really effective. Here we have three people interacting in different ways. One's using Kinect, one's using HoloLens and other one's using Virtual Reality.

Mixed Reality Rapid Value Roadmap
Paul Williamson37:38

So where we go from here? Well, we have, as Inter Reality Labs, we have what we call our Mixed Reality, Rapid Value roadmap. So if you're looking to explore how this technology can be applied into your business on a one-to-one basis, we have a little bit of a life cycle, which may come in handy. So the first one is what we call the Mixed Reality in a Day.

This is where our team can come to your site with a little bit of preparation, and we can actually show these applications working in your workplace. It can be working with some of your business problems and implementing some of your inspections, some of your workflows, actually on your equipment, in your customer environment, wherever makes the most sense for you.

So we can do that anywhere in the world. Following on from that, you'll be identifying the key use cases that are going to drive your business case. And those would generally result in approved Proof of Concept projects. So we can provide a fixed price structure for certain levels of Proof of Concept.

And we can help deliver those in a period of two to four weeks as a turnkey project to help accelerate your program. And from there we go into Pilot programs, which of course are going to be customized. But by then, you know what you want, you've been able to prove the value to your management. And what we're looking to do is now transition to Business As Usual.

And so with our global help desk, 24 hour support and our experienced platform teams and our ability to deploy quality Field Kits and accessories and hardware anywhere in the world, then we can help you get those Pilot programs off to the greatest chance of success and support you as you move into business as usual, and then also support you as you grow that technology into new areas within your business for maximum effect.

Rapid Value Roadmap - Mixed Reality in a Daqy
Paul Williamson39:36

So the mixed Reality in a Day is focused on your setting. It's prepared for you using your data. We try and identify the really high value, we've got the experience. So we're going to be using our experience to help you identify the areas in your business where you're going to get the highest value out of the effective technology of today, and also identify where it'll deliver value in the future.

We'll also help you gain the actionable insights that drive approvals of business cases. Help your senior decision makers make those decisions and then support your team as you employ this technology in your business.

Getting Started
Paul Williamson40:13

What do we need from you to start? We need to know who the company is and the background. Preparation can take a few days in this particular case, if you're in the UK or in some of our supported environments, we may be able to get some support from Microsoft which will help us accelerate and do some of this work for you using your data.

But don't let that limit you. We can deliver this pretty well anywhere in the world even if we have to do it by remote.

We'll deliver focused work with your senior decision makers in a day, and then deliver that rapid result.

And from there, we can move into a mixed reality Proof of Concept. So here we have packaged up just for ease of decision-making Remote Assist packages, Guides, and Mixed Reality, which combines the two and providing them as a fixed price service into your business. So you know what to expect and, you know what it's going to cost.

Thank You!

So that concludes the presentation for today. Thank you very much for your time. We'll be providing some extra contacts and resources linked to the website where you find this.

Should you require any further information, please don't hesitate to get in touch with us and we'll be very happy to help you explore your Mixed Reality journey. Thanks. Bye for now.