What Makes a Product Professional (in the age of AI)?
17 Nov 2024A professional is someone who makes a living doing something. In order to be a professional, you must perform that function with high enough quality, consistently, over time, fast enough to pay the bills. People pay for quality. The higher and more consistent the quality, the more they will pay.
If you can’t produce high enough quality work that people want to pay you for it, you aren’t a professional.
If you can’t do that work fast enough, then you don’t make enough of your work product to pay your bills, and you aren’t a professional.
If you can’t sustain that quality and speed for months and years, you are an amature, not a professional. An athlete who can’t consistently make free throws, throw fast balls, or make tackles game in and out season after season, doesn’t make the pros.
This is true for all professions. Lawyers, doctors, artists, athletes, military officers and product managers.
Another thing that is true of most professionals is that they learned their craft from another professional. And they practice for years to be good enough. For some professions it is mandatory that you get trained by an existing practitioner to become a professional. You cannot feasibly be trained as a lawyer by someone who is not a lawyer and you need to be certified by current lawyers if you want to represent clients in front of a judge.
Becoming a professional takes training and practice.
- Law school is three years long
- Medical school is at least four years plus optional specialty training
- Most pro sports players start playing their sport in grade school
- Artists make lots of art before they can do it full time, some go to art school, most study and appreciate the art of other artists.
- Ryan Niel, the leading bonsai artist in the U.S. spent 5 years as an apprentice to a Japanese master before becoming a professional himself. (The Beautiful, Brutal World of Bonsai) Before he could get that apprenticeship he spent more than a decade learning from anyone who would teach him in the U.S.
- Military officers go to training every 3-5 years to expand their understanding of the profession of arms
- Great product managers and leaders spend years doing repetitions of building, launching, and revising product. If we are lucky we work for great product managers who show us the way.
If you are a product professional you can help deliver products that add enough value to your company that they pay you to continue doing it.
- If you can’t ship product fast enough, you’ll be let go
- If you can’t ship products with enough quality that your customers will use them, then you’ll be let go
- If you can’t consistently ship products….you get the idea
So what does this mean for product professionals in the age of AI? I think LLMs might help you go faster, sometimes. You can tell an LLM to write your product brief, analyze your meeting notes, maybe even draft a strategy document. But I’m not certain these tools will make you a professional on their own.
LLMs have no sense of quality. They will produce the statistically most likely next word. And have demonstrated that those words won’t be the correct word, it might even be the catastrophically wrong word. You have to understand the subject to really know what your getting. Knowing your craft takes time and practice. We develop taste through repetition and feedback.
Already we’ve seen instances of AI products that are wildly insecure and enabling acting out child abuse fantasies (Hacked ‘AI Girlfriend’ Data Shows Prompts Describing Child Sexual Abuse) or providing an easy button for stalking and doxing (Someone Put Facial Recognition Tech onto Meta’s Smart Glasses to Instantly Dox Strangers). The statistically most likely next word is not what folks pay for. I don’t go out of my way to use average products, read average books, watch average TV.
If you want to make consistently great products you need to spend time learning what makes good products and what makes bad products. What are the failure cases, corner cases, abuse cases, non functional requirements and -ilities that make a good product. LLMs don’t have taste, you need to have that.
The final challenge with LLMs is that they have a consistency problem. You care ask them questions and get different answers on every iteration. Slight variations on prompt wording can result in wildly different outputs. Wildly different outputs is not what businesses expect out of their product leaders. Businesses and investors like predictable growth, not wildly varried results. No one wants a pitcher who can’t consistently throw a pitch in the strike zone or who’s fastball varies by 20 mph.
So there are no shortcuts to consistent quality. You need to practice your craft to know what
What are they good for?
There are things that LLMs are good at. There are tasks that they will help you go faster. But you will have to balance that with more critical thinking to read the output and make sure that it is correct. Let’s all continue to fight a war on drudgery.
There are probably places where consistently average is better that consistently sub par or wildly varried human generated content. IF you can validate correctness. If you are a below average writer, an LLM can probably help you be average. If you are an above averager writer maybe you’ll be pulled down to average. But not everything needs to amazing, some things just need to get done.
I think that there are probably opportunities to leverage LLMs as simulators that can accelerate your learning and craft. Maybe we can use LLMs to generate low fidelity prototypes and iterate on them, or critique them.
- Pilots spend time on simulators allowing them to practice many repetitions of emergency procedures in a very safe environment.
- The military runs war games at many different sizes from tabletop exercises to massive brigade sized games of laser tag in the desert to practice the art of war without fighting actual wars.
- Baseball players and golfers spend hours in the batting cage or driving range to have repetitions of hitting the ball until it is second nature.