There is an interesting scene in the TV show Silicon Valley where Gilfoyle, the team's lead sysadmin, tells the team their AI program is about to crack virtually every encryption protocol ever made. This includes P256, AES-256 (Military Standard Encryption), SSL, etc. As Gilfoyle states, this could be the "...the end of privacy".
Now obviously this is just a TV show and the danger was relevant to the script of the show. That being said this may already be happening.
I'm not one to regurgitate the "big brother" style statements about big data companies like Google. That doesn't mean I don't think Google is trying to understand me on a much deeper level than I would be comfortable with. Even companies that don't care have to care about your data because people have become the commodity. There's a quote that goes "If you aren't paying for the product, you are the product".
We already know Google has been training its own AI models for years now. There was even an article a couple years ago about the Google engineer who was fired after claiming the AI model he was working on had become sentient (After hearing his reason why I agree it is pretty close to genuine intelligence but not sentience). Who says they aren't working on a model to guess passwords. It may be bundled in a "Profiler" style AI model that looks at your life on social media and makes guesses about you. Judging your favorite color or favorite restaurant based on your posts. They could have it reading your Google account data for the types of places you are visiting or what companies you are clicking through on ad spots.
Even now they actually do these things in the name of "personalizing" advertisements and I wholeheartedly thank the EU for making it harder for them to do so, but they are still casting a very wide net across their users. I would say for every user who truly cares about their data there are a hundred who would give it up freely for the convenience of having personalized restaurant recommendations or a customized YouTube feed. It's not a stretch of the imagination that if Google knows you eat at Olive Garden Three times a week and always order the pasta they might guess pastalover123 as your password. This might sound like hyperbole but with computers able to make thousands of guesses every second it's not far off. Especially when we are automating OSINT gathering with tools like Maltego and their API tools.
Privacy is just different nowadays. We only started using middle names a few hundred years ago. There's even a joke about wanting to get a divorce in the twenties; "Just drive two towns over and go by a different name". Now everyone has so many different lives and secrets they are willing to share with each one. Seemingly everyone has an online account somewhere leading to the rise of the digital footprint. People are so much easier to find and get a hold of today than any other time in history.
It's become harder to keep secrets as well. You can't call each other securely any more thanks to devices like The Stingray used by law enforcement when approved via a warrant. You can text each other but numbers can be spoofed, the messages aren't always encrypted, and even the apps that do allow E2EE like Signal require both parties to have the service (Signal used to let you text non-Signal users but they removed it due to security). Even talking in person isn't safe. You never know if your phone or your acquaintance's phone has been tapped or is sending your audio to Google.
A lot of people freely give away their information too. When you sign up for Facebook they ask you for some very personal information. Many financial institutions even collect your social security number and your driver's license number through the internet. Sure it's secured via SSL but there are always vulnerabilities just waiting to be found. Many people basically hand over their identity to these hidden figures on the other side of the screen; giving them the building blocks to steal your identity.
In my opinion, thanks to A.I. this is all inevitable. Everything you have ever wanted to keep in the dark will be brought to the light. All of your cheating, affair having, birthday surprise keeping, mother's maiden name knowing, "I know what you did last summer" secrets will come to light.
The greatest part of all of this (in my opinion) is that when the dust settles and everybody knows what everybody else was hiding it won't matter. Sure, there may be a period there where people do get hurt, mentally and physically but everyone will be on a level playing field. Once we learn to be transparent with others in every aspect we have nothing to be angry at each other for. For many growth will be possible where there was only pain before.