AI Bridges Japan's Infrastructure Maintenance Gap: Civil Engineering Innovators - Morikawa Haruna

"RISING"

AI Bridges Japan's Infrastructure Maintenance Gap
Civil Engineering Innovator - Morikawa Haruna

Japan is home to some 730,000 bridges.

But gradual wear and tear increases the risk of collapse

during typhoons and other extreme weather events.

And in the next decade,

more than half of Japan's bridges will reach the 50-year mark

said to be their safe natural lifespan.

But major obstacles stand in the way of the regular inspections

that are key to proper maintenance.

Many bridges built during Japan's economic miracle

of the mid to late 20th century

are reaching the end of their lifespan.

And we simply don't have enough people to check them all.

That's in terms of bare numbers and the necessary expertise

which is actually declining.

Meanwhile, more and more bridges are getting older,

so the personnel deficit actually looks set to keep increasing.

And this fight against degradation is also required

to maintain safety in tunnels and other crucial road infrastructure.

Fukuoka is a vibrant city at the heart of the western island of Kyushu.

And a local business of just 40 employees has been in the spotlight

for developing new systems that ease the burden

of vital infrastructure inspections.

They're led by CEO Morikawa Haruna,

and her husband and co-founder Ayumu.

The construction industry faces personnel shortages and an aging workforce.

Our vision is to tackle those issues using technology.

As former development head with a local building contractor,

Ayumu brings technical expertise.

Our aim is technology with practical applications.

You could have this fantastic technology,

but if it's too expensive, it can't be used on the front line.

Our aim is solutions that can be used even in settings

where there isn't much budget available.

So how is the firm promoting new efficiencies

in infrastructure inspections?

We accompany their team on their five-yearly survey of a local bridge.

The bridge which forms part of a key commuter path

from Fukuoka's suburbs was built in 1980.

And, as well as the surface,

it's vital to examine the underside of the structure.

One of our main tasks is looking for cracks and so on

and measuring those to see how big they are.

Then, we also tap different spots with a special hammer.

We listen for any variations in the sound

that may indicate loosened concrete or other degradation.

These inspections are important

because if such structural issues are left to deteriorate further,

they could lead to serious accidents including collapsing bridges.

And because most bridge inspections involve heights,

the work can also be quite risky.

They begin with a preliminary visual inspection.

Any potential issues are marked with chalk.

These marks show where cracks are located.

They provide a clear visual guide of location.

And this code contains further details.

Next, they tap the concrete with a special hammer

to check for any interior issues.

And the inspection isn't solely concerned with concrete elements.

We also examine metal components,

the bolts and so on to check they aren't loose.

This time they do find an issue.

This bearing is badly corroded.

Early identification and repair of such failings

is key to maintaining safety.

The process we've seen so far is standard for most inspection contractors.

But it's the next step that sets Morikawa's company apart.

Ordinarily, the engineers would use their findings

to create a detailed repair plan on the spot,

and this would form the basis for vital repairs

to reduce the risk of serious structural failure.

But Morikawa's team has no need to create such plans on site.

Once their check is over,

they split up to take multiple photos of the bridge's underside.

Shooting these pictures,

which clearly capture the earlier chalk markings,

takes much less time than preparing a repair plan on site.

We used to take our hand-drawn plans like this one here

back to the office and use computer assisted design programs

to turn those into detailed repair schematics.

But at our company, we are now able to generate those schematics

based only on the photographs we have taken,

which overall brings a really big efficiency saving.

In the past, engineers would return to the office

and use computer assisted design or CAD software

to turn their hand-drawn plans into final repair diagrams.

Requiring specialist knowledge,

this painstaking task would often lead to countless hours of overtime

which in turn limited engineers' availability for on-site work.

But Morikawa's firm has developed a way for staff

with no engineering expertise

to create these diagrams in timely fashion.

So how do they go about it?

First, they upload the photographs of the chalk-marked bridge underside

into a special program.

Next, the program uses AI to arrange those pictures like a jigsaw

into a full view of the surveyed area.

It also scans the chalk markings

to highlight areas that are cracked or damaged.

These are then superimposed onto a digital map of the bridge...

before the operator keys in the code that accompanies the chalk markings,

and the specially trained AI system converts these into technical language

that provides precise details on the size and nature of each bit of damage.

This system enables even personnel

with no specialist civil engineering knowhow

to quickly create practical maintenance plans,

and all in just an eighth of the time required using older methods.

Morikawa's firm also creates plans from data

provided by other maintenance contractors.

Matsumoto Norihisa is the CEO of one such firm.

We had such a backlog of work

that we were having to turn down potential jobs.

Like, that's it, can't do anymore.

But now we're able to keep up with demand even in busy periods.

And besides our own company,

I think this system is positive for our industry as a whole.

So this groundbreaking system uses AI

to create maintenance plans that save aging bridges from disaster.

But what was the inspiration for Morikawa Haruna and Ayumu's efforts

to develop a system that anyone can use to create those plans?

The story began some four years before they founded their own company.

At the time, Haruna was a full-time housewife,

while Ayumu was development head with a building contractor.

A friend suddenly wound up on her own with two kids to raise.

She told me how tough it was to find work as a single mom,

but I'd also heard from my husband who worked for a contractor

about personnel shortages in the civil engineering sector.

So it all started from the idea

that we might be able to address all that somehow.

Back then, it just wasn't possible

to offer work to staff with no special experience.

I'd had the idea for an AI system

to automate creation of maintenance schematics for bridges and tunnels,

but there was so much data involved

that I just couldn't get clearance for that at my old company.

The idea just got buried as impractical.

The original concept was to help women like my friend find work.

And we decided one way to get things moving could be,

for me as a housewife, to perform the preliminary data input we needed.

The pair got down to work.

Ayumu programmed the basic system, while Haruna trained the AI.

That meant keying in an amazing 50,000 data points

relating to Japanese transport infrastructure maintenance standards.

But it was a much bigger task than she expected.

It was me who suggested that inputting the data would get us there.

But in truth,

I didn't tell her just how much data there was

or how tough it was going to be.

I didn't really get how tough it was at the time.

I just got on with it.

But thinking back, it was a heck of a task.

There were quite a few glitches too.

When we ended up losing all the data that she'd already keyed in,

I really just felt so bad for her.

That was one of the difficulties we faced.

What kept me going was the thought that when we were done,

we would be able to give my single mother friend some work.

And after four years of work,

they had a system that would let even novices create

digital maintenance plans from photos taken on site.

Then, in March 2018,

they founded their own company with Haruna as CEO.

But the AI system still lacked accuracy

and struggled to identify every piece of damage shown in photos.

To solve this, they incorporated a feature

that let users trace over the chalk markings.

And by rolling out a system that enabled single parents

and people with disabilities to work from home,

they attracted attention as a social business, even appearing on TV.

It's quite fun.

Even a child could do it.

Little more than six months after its founding,

the company's efforts to solve issues in the fields of civil engineering

and workplace inclusivity saw it selected to represent Japan

at the world's largest social business summit held in Germany.

Then in 2019,

they won recognition at the Japan Entrepreneur Awards

and their expanding reputation began to attract business

from major Japanese companies.

Nowadays, besides creating maintenance plans,

they conduct bridge and tunnel inspections

all over western Japan and beyond.

Our inspections are based on a firm grasp of the needs

of front-line field work.

Our staff engage in development work

while also providing support for front-line maintenance checks.

Meanwhile, our front-line inspection engineers also help with development.

That's how we try to break down silos within our company.

But across Japan, besides bridges and roads themselves,

another significant challenge comes in the form of streetlight inspections.

Components falling from lampposts are now a common cause of accidents,

while aging examples can also be at risk of toppling

in typhoons or strong winds.

And while inspections are moving forward nationwide,

the sheer number of units presents challenges

in terms of budget and personnel.

In response,

Morikawa Ayumu is leading development on a new project,

a streetlight inspection robot.

They're currently working with Fukuoka City Council

for practical trials on the city's many lampposts.

We join the team for tests in the parking area of a local park.

First, they set up the robot.

Then the trial gets underway.

The robot has tires that grip the pole as it climbs.

A key feature is the top-mounted camera.

This provides detailed images of the pole

throughout the robot's ascent and descent.

The footage is relayed in real-time to a mobile interface

that lets operators check for rust and so on.

Meanwhile, two drone-like propellers provide lift and maneuverability.

It can even handle curves.

Another built-in tool replaces the traditional handheld hammer tap test.

The tires are equipped with vibration sensors that collect sound wave data

providing a detailed visual guide to potential corrosion and damage.

Ayumu explains a key development challenge.

Lampposts tend to taper as you go upward,

so the device needs to adapt to changing thickness.

We do that by adjusting the aperture

but it was a tough nut to crack.

And though the device employs drone-like features,

the constant contact of the wheels to the pole

sees it legally classed as a robot,

circumventing Japan's strict flying regulations.

And there's another key advantage.

These checks typically use an articulated aerial platform,

but this eliminates the need for that

which also means there's no need to obstruct the road and divert traffic.

Traditional lamppost inspections require large teams.

First, you need an aerial platform truck and crew

along with attendants to control the oncoming traffic.

In all, this typically adds up to a team of four or five.

But the new robot eliminates the need for a platform,

saving on personnel, time and, significantly, cost.

And the Fukuoka City authorities have high hopes for this new technology.

In Fukuoka City we have around 38,000 streetlights,

so efficiency savings would mean

we could check more in a given timeframe.

And these robots would eliminate the need for us to use trucks,

and that would remove the risk of causing traffic jams.

The next challenge is to make sure we keep within budget.

Then I think they could bring major benefits to our city.

But one major challenge remains for these inspection robots.

Some lampposts have signs or cables attached

which prevent the current robot from climbing up.

It's a really thorny issue.

In response, the team has also been working on an alternative approach

using a globe cage design

which is mounted on a pole and maneuvered by hand.

This can be raised past obstacles.

There's no separate controller.

It has sensors where the pole meets the cage

which work in whichever direction

the operator wants to guide the device.

Today is a preliminary test.

This device is also fitted with drone-style propellors

providing extra lift that enables operation

by less physically strong individuals.

And as well as a camera to observe the surface of the lamppost.

Tapping the cage against the post provides vibration data.

It's no good.

Can I put it down?

But on windy days like today,

it can be difficult to maneuver the device

as desired to higher-up locations.

With trial and error,

you're always going to end up with plenty of errors.

We just focus on making things and gradually ironing out the issues.

If you come back in a year, we'll have something completely different.

They're also working on an experimental system

to further improve the efficiency of streetlight inspections.

It uses a front-mounted camera.

Once it's connected to a computer, they set off.

The system is designed to inspect lampposts

simply while driving around town.

The front-mounted camera shoots footage of the roadside,

while an AI system detects lampposts

and scans them to check for details

like surface corrosion and missing bolts.

The idea is to improve efficiency by spotting potential issues.

Then we can come back for a closer look later

with our climbing robot or pole-mounted device.

Today, we join Morikawa Haruna on a visit to a local facility.

This organization is focused on assisting workforce participation

by people with learning disabilities or mental health challenges.

They receive computer-based work experience

that equips them for placements with companies.

For today's task,

these participants are helping to train Morikawa's AI system

in image recognition.

For example, when checking for loose bolts,

feeding the system as many images of bolts as possible

helps to boost accuracy.

But until now, such training has always been a job for experts.

Morikawa's firm has developed a system

that lets anyone perform this task.

All that's required is to drag and drop frames

around any bolts in the picture.

This task is a lot of fun.

It's a simple task,

but personally I find it very enjoyable

to figure out ways to improve my own speed and efficiency

as I proceed with the work.

Morikawa is actually here today to discuss recruitment.

She already employs three graduates of this facility

and is hoping to take on one more.

We arrange various kinds of work,

but it's great for trainees like ours to experience tasks

that bring social value by improving infrastructure and so on.

So we're very grateful to Morikawa-san's firm.

From the single mothers who inspired us to found this company,

to others who want to work but find opportunities limited,

our aim is a system that lets us provide them with work.

And as our workload increases,

we want to provide opportunities for people with no experience in our field

and more and more people who have struggled to find work.

That's our goal.

And this husband-and-wife duo have various plans

to build on their work solving issues in the civil engineering sector.

Our aim is robots and AI

with practical applications in front-line fieldwork.

And moving forward, I'd like to propose robots capable

of inspecting large facilities like nuclear and other power stations.

At present, our activities are focused on Japan.

But we're planning to expand internationally.

By leveraging that potential for anyone to perform these tasks,

we hope to provide working opportunities for people around the world

without the need for Japanese language skills.

So this company that began from an urge to help single mothers into work

is now preparing to expand overseas.

But what traits do this power couple see in each other

as the bedrock of the firm's success?

She's a fantastic communicator.

Personally, I get nervous when speaking to the CEOs

of multi-billion-yen companies,

but Haruna takes all that in her stride.

It's very impressive.

Of course, he brings the knowhow and experience

from his background in development.

But he's also great at spotting

what approaches might complement each other on the front line.

I think that's his major strength.