What I’ve learned working on AI/ML projects

Before coming to work at RomSoft, I had been studying artificial intelligence (AI) and machine learning (ML) for a long time. I used to participate in various AI competitions and even won a few prizes. This was back in 2003-2005.

In 2007 I organized a talk with my colleagues about AI and concluded that we could get involved in this kind of projects if the opportunity presents itself. Well, it did, 10 years later, and our managers, knowing my interest in the field, asked me if I was interested to join in.

Obviously, I was happy to accept the challenge. We’ve started with some small scale projects, with the objective to take small steps in exploring the artificial intelligence and machine learning areas, to acquire skills and capabilities, and to build a project portfolio.

What is, really, artificial intelligence?

Artificial intelligence is the simulation of human intelligence processes by machines, especially computer systems. Specific applications of AI include expert systems, natural language processing, speech recognition, and machine vision. In a larger sense, AI is supposed to perform tasks that not only are usually done by humans, but also require human intelligence and discernment.

While the general feeling is that AI and machine learning are very present on today’s public agenda, when you actually work in the field, you start to realize how distorted perception from reality really is.

As members of the general public, our imagination is fed with what we see in the movies. Humanoid robots looking like us and talking like us, AI software taking over the world, or leaving us without jobs.

But in reality, AI represents an automated interpretation of statistical data in order to optimise some parameters. At this moment, we can confidently say that machine learning is basically data science.

In order to launch an artificial intelligence/machine learning project, you need data analysis, you need people to perform that analysis, and you need to invest large amounts of money with a 60% failure chance.

Working in AI/ML can mean a lot of things

Working in the AI/ML field can require a different way of thinking. As I explained before, you work with a huge amount of data. On the CNN (convolutional neural network) preparation part, I may be less fit to work than, let’s say, a more artistic person, as CNN is an AI algorithm that is most commonly applied to analyze visual imagery.

Imagine the way insects see the world. The same with AI, you need to split the image in small squares and then reunite them. A more visually inclined person may recognize more rapidly the differences between images, if, for example, looking for texture or color differences.

Then, there is the data science area, where you need somebody who’s more analytical. A person who likes to analyze and work with big amounts of data, who can organize and prepare data.

The programming skills required are a small part – here we work with the classical algorithms, an input and an output.

And there are also the mathematical skills that you need to use a lot in the research part, where you need to correlate the data.

90% of the time spent developing an AI based project is consumed in negotiations to obtain better, more relevant data.

I can give you an example from a project I worked on.

I was trying to figure out, from a range of different images, where, in the photos, there was a human figure. But each picture was taken with the same background that included a green fence.

The program is building some kind of network and tries to see the most common thing for all photos. And the algorithm rendered as “true” all images containing the fence. Of course, it was not the answer we were looking for.

This shows how important the quality of data is. In this situation you turn to the client and let them know they need to redo the entire photo archive. This means more effort and more costs.

Speaking from my experience so far, 90% of the time spent in developing an AI based project is consumed in negotiations to obtain better, more relevant data. The development phase is the rest of 10%. That puts a lot of effort on the client.

AI vs. world

Who can work in an AI based project

The technical background that a programmer needs to work in AI related projects is not very different from regular projects. A strong taste for mathematics, complemented by a natural interest for statistics are a great plus. If, on top, you are gifted with some sort of visual thinking, then you and AI are a perfect match.

Of course, depending on the project, things can get very specialized. You cannot be a Jack of all trades all the time. In the automotive industry you may need more statisticians.

To recognize defects or objects in photos you may need graphic designers and visual thinkers. And this is where the negotiations start, and probably a business analyst may have a contribution to translate things from one side to the other.

The skills you may need to work in an AI project are evolving (and sometimes in surprising directions). Some data scientists even consider that the ability to make good PowerPoint slides may be more important than the ability to use the most sophisticated deep learning models.

What are the real challenges in AI today

Locally, we are still taking small steps. There is only one company in Iasi that specializes in AI and machine learning, and we collaborated with them last year on one project and hope to do it again in the future.

But otherwise, everybody is trying to allocate an exploratory budget, while working on other things. So we can say that we’re all facing the same types of challenges. At country level the situation is not that much different. There are still very few companies that are profitable only with AI projects.

Given that these projects have a strong research component, we also started to explore the EU funding programs for developing AI and ML projects.

Last year we wrote two projects but they didn’t come through. This only motivates us to make more efforts and try again. And it is encouraging that more funds are flowing into AI.

Through the Digital Europe and Horizon Europe programs, the European Commission plans to invest €1 billion per year in AI. It will mobilize additional investments from the private sector and the Member States in order to reach an annual investment volume of €20 billion over the course of the digital decade.

This is good news since at EU level, there is a shortage of specialists in AI and ML related fields, and a 5 to 10 billion EUR investment gap.

Ethics, a challenge we can’t ignore

Science was never requested to answer moral questions in all history. This job was for philosophers only. But with the raise of AI and algorithms, the situation is about to change. For example, in the case of a self-driving car, the software controlling the car should have embedded the answers to some moral issues.

The classical example is who should be saved from an impact, the pedestrian or the driver, if given a situation where it is impossible to save both? It is clearly a very complex issue that engineers are now bound to answer.

This is an extreme example, but the truth is that all forefront technology domains, like AI, blockchain or bioengineering are, at the current moment, highly unregulated industries and could raise many ethical problems. Maybe philosophers, psychologists, or ethics professors could also make an important contribution to these areas, alongside engineers and technical people.

What is (y)our place in the AI (r)evolution

Beyond the concern that is usually advanced by the media, that AI and robots are going to take our jobs and render us humans useless to society, I would speculate that many other jobs will emerge from this technological revolution, like in all past revolutions.

The AI movement will touch the majority of fields and will need the contribution of all kinds of specialists. With these worries addressed, we can now focus on more practical issues.

It takes 3 or 4 engineers to work for one year, round the clock, to bring some results in a major AI project. And they can’t be junior engineers. You need to dislocate them from a current project that is producing money for you now. And the market place is limited when it comes to experienced specialists.

So if we, the EU, want to fill in that investment gap, we should start thinking long term. And if private companies are willing to invest and explore the possibilities of the domain, states and the European Union should do their part too, and maybe, if we can meet halfway, we can work towards some relevant results.

Whether we like it or not, even if at this point we are, in terms of the development of artificial intelligence, at the mouse-brain thinking level, evolution will probably be exponential, and I expect that in 10 to 15 years we will reach technological singularity. And we can’t act surprised when this happens. It’s up to us to be part of this movement and to bring our contribution.

I would love to hear your thoughts on the subject of working in AI / machine learning. What does it mean for you? What skills did you have to develop?