It's exactly the direction machine learning is heading and the current progress of AI is clear. Imagination bears little relevance here really going forwards, whilst right now it may matter, that won't be the case soon enough as exploit finding is just much faster with AI than it is with human imagination.It's really not. AI lacks something that all good hackers have - imagination.
Cloudflare have a great article on it:
As artificial intelligence advances by leaps and bounds, the possibility arises of training this technology to find vulnerabilities even more effectively. In fact, in 2023 the US agency DARPA announced a program called Intelligent Generation of Tools for Security — INGOTS. (DARPA, notably, was the agency that created ARPANET, the precursor to the Internet.)
The program "aims to identify and fix high-severity, chainable vulnerabilities before attackers can exploit them" by using "new techniques driven by program analysis and artificial intelligence to measure vulnerabilities." INGOTS looks for vulnerabilities in "modern, complex systems, such as web browsers and mobile operating systems."

theNET | AI-powered vulnerability detection
Can AI be used by attackers to detect vulnerabilities in software? Discover how to protect against AI-assisted vulnerability detection with Cloudflare.
So yes, just as suspected, right now humans still need to review AI's findings to sanity check them, but that won't be the same landscape as the technology evolves faster and faster every year. It's pretty clear that human intervention will be reduced at each stage as machine learning gets to the point of singularity said to be around 2035-2045).
Last edited: