Download PDF version

Combining thermal imaging and augmented reality (AR) enables firefighters see through smoke, in effect enhancing their vision in the life-threatening environment of a fire. AR capabilities can be deployed in a visor attached to a helmet, and an affixed thermal camera captures the images.

The most recent prototype of such a product is a robust helmet design that withstands rough treatment. The system also includes software processing that augments thermal images to enable firefighters to see the outline of objects more clearly, thus enabling their detection in the field.

augmented reality

Firefighters can see through the visor/screen, or they can view the thermal image video live overlaid onto their vision. A switch on the side of the helmet allows them to switch among various modes – gray scale, RGB mode, fusion mode, etc. National Fire Protection Association (NFPA) standards specify what information appears on an AR screen. In general, mostly emergency alerts appear so as to simplify the view.

The technology began as a project at McMaster University in Hamilton, Ontario, Canada, to combine augmented reality (AR) with thermal imaging for use by police and fire departments. The project has evolved into the startup company Longan Vision.

Live fire situations

Funding has been provided by Defense Research and Development of Canada (DRDC) and private sources

Development began in 2018 and has continued for two years through six generations of prototypes. The goal throughout the iterations has been to make sure the product is sufficiently robust for use in the field and will hold up to rough usage by firefighters, according to Enzo Jia, CEO and founder of Longan Vision. Funding has been provided by Defense Research and Development of Canada (DRDC) and private sources, and the project has progressed in partnership with industry manufacturers including helmet and apparatus companies.

After two years of development, the company is on the verge of testing the product in live fire situations. Pilot testing will begin in January 2021. The device will begin pursuing NFPA certification next March, and hopefully the company will be shipping products by June or July 2021.

Local fire departments

Longan Vision has depended on feedback from fire departments throughout development of the product. They currently have partnerships with more than eight local fire departments, three in the United States. They meet with fire chiefs and firefighters and ask for their honest opinions on how the device operates, whittling down the good and the bad.

Feedback from the departments included the need for hands-free thermal imaging and a system that displays all needed information in their view. The devices have also been tested by fire departments in Tokyo, which has close relationships with Longan Vision’s fire apparatus partners there. “We have tried to work very closely with firefighters,” says Alex Shortt, Longan Vision’s CTO. “We are a group of engineers, but we need firefighters to help us develop a good product. We keep a firefighter focus in all the work we do.”

Virtual reality visor

The AR device can connect with smart phones or other Internet of Things (IoT) devices to add additional information

There are three possibilities for taking the technology to market. One is to work with helmet manufacturers to incorporate the AR technology into helmets. A second possibility is to work with fire apparatus companies to provide the technology as part of a broader ‘system’ of capabilities surrounding modernized apparatus.

The third option is to equip the technology with a ‘universal mounting system’ – a series of bands and/or straps that allow the camera and attached virtual reality visor to be installed onto an existing helmet. The AR device can connect with smart phones or other Internet of Things (IoT) devices to add additional information to the system and visor display screen, such as the heart rate of the firefighter, for example.

Incident Command Center

Firefighters can communicate among themselves using a Mesh network, or alternatively, a 5G network if available. Each firefighter’s unit can also communicate with an incident Command Center. Incorporating VR into a robust helmet is preferable in the fire industry to using an off-the-shelf AR device such as Google Glass or Microsoft HoloLens, says Jia.

The AR view can be recorded as video to an SD card in the helmet, which can hold two hours or so of footage

These devices are not designed for the rigors of the firefighting environment and therefore must be worn inside a self-contained breathing apparatus (SCBA) mask. However, in that scenario, if there is a device failure, removing the device is more complicated, as is the process of handing the device over to another firefighter if someone ‘steps in’ to an emergency situation. The AR view can be recorded as video to an SD card in the helmet, which can hold two hours or so of footage. However, the demand for such a feature is mixed among fire departments.

Early detection method

A fire chief might want the recording to use for training purposes, but a firefighter might not want the video to be used to judge the potential liability of his every action, which could be open to legal scrutiny or even lawsuits. The COVID-19 pandemic has had an impact on Longan Vision as they look to launch their new product, including a tightening in the procurement market.

On the positive side, the company has adapted the technology into another product, Gatekeeper, that uses thermal imaging to measure skin temperature as an early detection method for elevated temperatures. The system has an accuracy of plus-or-minus 0.3 degrees Celsius and can measure a maximum of five people at a time. The wall-mounted, non-contact unit captures a face image of people entering the video frame and displays each face alongside their skin temperature.

Share with LinkedIn Share with Twitter Share with Facebook Share with Facebook
Download PDF version Download PDF version

Author profile

Larry Anderson Editor, TheBigRedGuide.com, Notting Hill Media

In case you missed it

Securing The Next Generation Of 911
Securing The Next Generation Of 911

While conducting research for my 2021 Wisconsin Public Safety Commission (WIPSCOM) conference presentation, it became immediately clear that securing the nation’s public safety answering points (PSAPs) is no longer just an IT challenge. Shifting from an analog to digital operating environment — the next generation of 911 — will require strategic investments into three key areas: people, processes and technology. As call center technology evolves nationwide, the need for greater cybersecurity in the public safety space has never been more important. Internet connected systems Traditionally, PSAPs received calls over analog telephone networks consisting of copper wire transmission lines and dated cellular networks spanning a smaller area in close proximity to call centers. With the introduction of next generation 911 and the accompanying digital telephone networking services, the exposure of call center networks to would-be attackers has exponentially increased, allowing potential access from anywhere due to internet connected systems. A good analogy is thinking of the points of entry into your home. The legacy method allowed two entries into the house — the front and back door. With the introduction of digital networks, there are now more doorways into the home or call center, signifying a greater need for security and employee awareness of threats. Many of these recent attacks target people using a technique called social engineering First responder organizations Since 2019, there have been approximately 300 cyberattacks impacting local government agencies, including police stations, emergency dispatch call centers and first responder organizations. 125 of these attacks specifically focused on public safety agencies such as firefighting or EMT stations, with attacks reported in all 50 states. More recent examples show that cyberattacks focusing on our first responders are increasing at an alarming rate. Many of these recent attacks target people using a technique called social engineering. This is when attackers attempt to trick victims through telephone calls and/or emails to assist the attacker in introducing viruses to the network, provide sensitive data or share usernames and passwords to achieve their criminal motives. Cyber criminals' primary objective is to use social engineering techniques to achieve a much more serious attack: ransomware. Critical computer systems Ransomware is a type of malicious software (malware) that prevents access to sensitive files Ransomware is a type of malicious software (malware) that prevents access to sensitive files, data and critical computer systems using encryption that only the attacker can unlock. Victims must pay a random sum of money, usually in an untraceable cryptocurrency, to the attacker who promises to decrypt data once they receive the funds. A look at attacker motivations can help us all understand — and mitigate — the threat to our first responders. Here are three primary reasons why cybercriminals target public safety answering points: Monetary gain: Infecting a PSAP with ransomware can lead to significant payouts in order to restore first response services. Disruption of services: Shutting down critical services can put threat actors in the public eye while also playing a major role in multi-stage attacks. Cheap thrills: Attackers and, at times, even misguided amateurs can target critical services for notoriety or social standing. Mitigating cyber risk The human element, actions or inactions played a direct role in 85% of data breaches Regardless of the motivation, the outcome is generally the same: a disruption of first response services that are critical to protecting our communities and families. According to Verizon's 2021 Data Breach Investigations Report, the human element -- or people’s decisions, actions or inactions -- played a direct role in 85% of data breaches. As cyber threats targeting PSAPs and first responder teams continue to grow in number and severity, addressing the threat through employee awareness and education is a good first step in mitigating cyber risk. Here are four steps any PSAP can take now to assess and mitigate cyber threats targeting their organization. Security awareness training Educate employees with security awareness training - Ongoing security training efforts should occur at a general level for all PSAP employees, followed by more targeted, role-based security training for key roles and departments such as call center managers, dispatchers or those with access to sensitive data. General security awareness training efforts should focus on broad but relevant security topics employees are likely to encounter, such as how to identify a phishing email. Security training programs should occur at least annually, and training content reviewed semi-annually Role-based security training efforts should go one step further and include topics like how management should respond to ransomware payment demands or how to verify the identity of external callers asking for sensitive information or urgent payments. Security training programs should occur at least annually, and training content reviewed semi-annually to ensure completeness, accuracy and relevance of training content related to your operating environment. Physical building access Verify and strengthen employee access controls - This includes physical building access and logical access to any information or computer systems your organization operates. Most organizations have several internal or external users such as vendors, cleaning companies and other organizations who come into contact with the offices or other physical locations, increasing the risk of theft or unauthorized access via impersonation or tailgating attacks. Ensure exterior locations are sufficiently secured via electronic badge access or a minimum of key access with code entry. First responders and public service agencies should train employees to visibly display employee badges and report infringements to management in the event an attacker infiltrates the building. Multi-Factor authentication External visitors should be required to announce their arrival in advance to the organization External visitors should be required to announce their arrival in advance to the organization, enter through designated areas, check-in with a receptionist or direct contact, log their entry, show identification and wear a clearly identifiable visitor badge. Access to computer systems that contain sensitive data such as employee records or connections to other state and federal agencies should be secured via multi-factor authentication. Multifactor authentication is a security term referring to authenticating a computer system using several factors, including something you know (e.g., username or password) , something you have (e.g., smartphone) or something you are (e.g., fingerprints or voice pattern). Using two or more factors when accessing a computer system is crucial to keeping the cybercriminals out! Federal threat intelligence Leverage free resources to mature your cybersecurity posture - First responders and public service organizations have many free cybersecurity resources at their disposal. This includes federal threat intelligence via security advisories, which outline vulnerable software or hardware products they use, and direct consultation services from cyber response teams local to the area, which are taxpayer funded. The US-CISA also provides regional consultation services to assist all local government agencies Every first responder and public service organization should consider becoming a member of a relevant Information Sharing and Analysis Center (ISAC) such as the Multi-State Information Sharing and Analysis Center (MS-ISAC), as they provide free threat intelligence services and consultation resources to help boost cybersecurity. The US-Cybersecurity and Infrastructure Security Agency (US-CISA) also provides regional consultation services to assist all local government agencies in maturing their cybersecurity posture. Public service organizations Hire external security firms to identify and correct weaknesses - To the extent allowed by budgets and personnel, first responders and public service organizations should hire external security or audit firms to assess the state of their cybersecurity practices and posture. These firms specialize in security best practices and assess security controls' adequacy across a wide array of organizations. It is often useful to bring these firms in for a fresh perspective on how the organization operates and its vulnerabilities. These engagements are typically performed annually and focus on core computer systems and business processes that involve sensitive data.

Using State-Of-The-Art Technology To Prevent And Put Out Wind Turbine Fires
Using State-Of-The-Art Technology To Prevent And Put Out Wind Turbine Fires

As more and more countries in Europe and North America commit to net zero, a key strategy is replacing old fossil fuel-driven forms of power generation and replacing them with renewable energy, such as wind turbines and solar panels. The wind industry has seen a particular boom, with tens of thousands of new turbines installed every year across the globe. However, like any other heavy machinery, wind turbines can catch fire due to mechanical or electrical failures. These fires can have impacts beyond the turbine if there is secondary fire spread to surrounding lands, resulting in potentially catastrophic loss. Without this technology in place, a single fire could cost $7-8 million and cause substantial downtime. The time is now for the industry to use all available technology to prevent these incidents and reduce the risk of fires spilling into the environment. How do wind turbine fires start? Wind turbine fires can catch fire due to external causes, such as lightning strikes, or internal causes, such as mechanical or electrical failure resulting in sparks or heat in the nacelle. Most nacelle fires start at one of three points of ignition – converter and capacitor cabinets, the nacelle brake, or the transformer. Nacelle brakes are used to stop the turbine’s blades from spinning in an emergency.  Converter and capacitor cabinets and transformers are necessary for the turbine to generate power and transform it into a voltage that can be exported to the grid. An electrical fault at either location can produce arc flashes or sparks, which can ignite nearby Class A combustibles, like cables, plastics, or fiberglass. Nacelle brakes are used to stop the turbine’s blades from spinning in an emergency. The brakes can cause turbine fires, albeit due to sparks from mechanical stress and friction rather than electrical failure. While some turbines have been designed with safer, electrical brakes, mechanical brake systems are often used as a backup in the event of power or control failure. These ignition points are all necessary for the safe generation of electricity from the wind, and cannot simply be designed out. As such, wind farm owners and operators must be ready to deal with fires when they spark. Why are wind turbine fires hard to fight? Modern wind turbines often exceed 250 feet in height, while most ground-based firefighting can only reach up to 100 feet. A team sent up-tower to manually fight the fire would constitute a major health and safety risk, as turbines have limited space and escape routes – putting employees not only in direct contact with fire but at risk of being in the turbine if it collapses. As such, when turbines catch fire, they are often left to burn out, with firefighters’ efforts focused on preventing the spread and clearing the area as fiery debris falls. This results in irreparable damage to the turbine, necessitating its replacement. What is the cost of a wind turbine fire? The cost of replacing a burned-out wind turbine depends on a number of factors. First and foremost is the size and initial cost of the turbine. Turbines with more than 3MW of rated capacity can cost between $3-10 million to install during development. Replacement turbines can often cost even more, as manufacturers are likely to charge more for individual, one-off installations. Another key loss is business interruption, or how long the turbine was offline – and therefore not generating revenue. The average loss due to a turbine fire was estimated by insurance company GCube to be $4.5 million in 2015. As turbines have grown larger and therefore more expensive to replace with greater losses in revenue, we expect a fire to cost anywhere between $7-8 million for new models. How can turbine owners and manufacturers manage fire risk? Firetrace’s system is designed with flexible Heat Detection Tubing, which ruptures in response to extreme heat or open flame Turbine manufacturers are already taking steps to “design out” fire risk in turbines. For example, lightning protection systems on turbine blades safely re-direct the surge of electricity away from cables, while condition monitoring systems can identify whether a component is overheating and likely to catch fire. In order to put out any turbine fires that do start at their source, turbine owners and manufacturers can install automatic fire suppression systems at common points of ignition. Firetrace’s system is designed with flexible Heat Detection Tubing, which ruptures in response to extreme heat or open flame, releasing a clean suppression agent precisely at the source of the fire before it can spread. Wind farm owners who have taken a more proactive approach to manage risk via fire suppression systems have been able to snuff out fires before they can spread throughout the turbine or into the environment. By investing in the latest technology for fire suppression, owners and operators have avoided the worst-case scenario, saving millions in operating costs.

PAMS Software Promotes Accountability Of Fire Service Responders
PAMS Software Promotes Accountability Of Fire Service Responders

The fire service has always struggled with maintaining accurate accountability of personnel who are responding or operating in emergencies. Lack of firefighter accountability is often cited as a contributing factor in Line of Duty Deaths (LODD). Compounding the accountability challenge are volunteer responders who can be coming from anywhere, with some going to the station and others going direct. The existing accountability tools and processes were unreliable and failed when needed the most. Need for reliable and accurate system As a firefighter and Incident Commander, Justin Brundage witnessed firsthand the data gaps of the tools and processes commonly used. A reliable and accurate system was needed in the fire service to avoid unnecessary risk to responses and responders. The intuitive process fits within an existing response workflow and provides an end-to-end solution  Seeking to address the problem, Brundage co-founded Incident Management Technology, whose Personnel Accountability Management System (PAMS) software is a solution for maintaining accurate and reliable firefighter accountability. The intuitive process fits within an existing response workflow and provides an end-to-end solution for firefighter accountability. The software was developed to solve operational gaps in emergency response and to help departments operate more effectively and safely. Real-time operations With the PAMS system, all personnel can see the available, deployed, and responding staff and resources in real-time on a mobile app or web browser. Responding apparatus are also viewable in real-time, including all the personnel on the apparatus. At an incident, the software tools simplify the accurate tracking and management of all personnel on the scene and enable a shared common picture of the who, what, and where of all responders at all times. PAMS gives department members and officers the information they need in real-time to optimize their responses. “We do this by sharing availability and response information throughout the department on a smartphone app,” says Brundage. Operational safety In addition to the improvement in operational safety that agencies get from PAMS, the software also improves response. “When all responders can see the other responders’ destinations and estimated times of arrival (ETAs) they can adapt and optimize the response efficiency by responding where they are needed most and not duplicating unnecessarily,” says Brundage.  PAMS software functions as an electronic equivalent to tag-based systems, which are ineffective, cumbersome, and error-prone. The key difference is that, by being electronic, the “accountability” information is viewable to anyone connected to the agency in real-time, regardless of location. Computer-aided dispatch (CAD) The software manages the responder throughout the lifecycle of the emergency response New incidents are sent to the responder mobile app automatically from computer-aided dispatch (CAD). Responders mark up if they are responding, and the system calculates and shares each responder's destination and ETA. The software manages the responder throughout the lifecycle of the emergency response. The entire department can see who is responding, who is assigned to each responding apparatus, who is operating at the incident, and where they are operating. Because this is an electronic process managing the personnel, is much easier with timers on task activities, and a simple and quick participatory action research (PAR) process. Fits in emergency workflow PAMS software is designed to fit into the existing workflow of an emergency response. “As responders ourselves we understand the burden of adding more operational requirements to the already chaotic moments of response and incident mitigation,” says Brundage. PAMS was built to work effectively on the equipment that is in many cases already deployed and installed in the response apparatus. The mobile app is available for iOS and Android and is used by the personnel responders, and then the web app is browser-based and can run on a browser window on tablets, mobile computing devices (MDCs), and laptops. Affordable, But has a lack of awareness In rolling out the product, awareness has proven to be a challenge for Incident Management Technology. “As a startup company most agencies that would benefit from the system aren’t aware that a solution like this even exists,” says Brundage. The system is expanding features and functionality to maximize incident response effectiveness The system is expanding features, functionality, and integrations rapidly intending to build an affordable solution for all fire departments to minimize their operational risks and to maximize their incident response effectiveness. Benefits of the software “We are currently having success with organic growth and the network effect,” says Brundage. “Our current customers are showing the system and validating the benefits to other agencies local to them, and we are increasing our awareness that way every day.” He adds, “We love doing web demos and talking to fire and EMS departments. Most fire departments have the same operational challenges, and the feedback we receive from customers and prospects is what we use to drive our next phase of software development.”

vfd