While LLMs can be incredibly helpful for certain types of writing tasks, their effectiveness is limited when dealing with more subjective or creative forms of writing particularly if I don't have the expertise and language to provide nuanced guidance.

As I've written before, I've found using an LLM as a writing buddy to be incredibly helpful. The feedback on the flow of my writing, transitions, intros and conclusions has really helped me produce and publish on a schedule that I just wouldn't have been able to before. To be fair, sure I could have published if I had no standards. And maybe in the future after I've learned from its coaching I could. But starting from zero to daily, it's enabled me to publish at a pace and quality I wouldn't have been able to achieve otherwise. However, it seems as if I'm most productive when I understand enough about what I want to create that I can explicitly guide it toward success. When I don't know how to do that, it can feel like I'm just shaking a magic eight ball repeatedly and hoping for my preferred answer to show up.

I'm well into my 30 days of publishing and I went into the archives for what I might publish in the future. I found a little narrative I'd written about a cascade of parental drama 5 years ago trying to get my 2 young kids into the bath tub. I asked Claude for advice using my existing prompt for writing feedback and I did not like the feedback I got back. Not that it hurt my feelings but the recommendations didn't seem to improve my story. Reflecting on this experience, I considered several explanations for the LLM's limitations.

One possibility is certainly that I haven't figured out how to use the LLM effectively to set it up for success in helping me improve this type of writing. That's almost certainly true to some extent. It was my first attempt. However, another possibility is that it's much harder to explain the type of help you want in something that's more subjective. I circled back to try again and the results were a little better but it still didn't feel as powerful as in other scenarios.

My theory is that based on how LLMs work, essentially probabilistically guessing at what a good answer might be, it defaults to offering up the most popular styles that your submission can fit into. If the NY Times represents the majority of it's training data, it's going to try and help you by working your submission into the form of the NY Times. You can get specific and make adjustments but you need to language and understanding to guide it.

If you are knowledgeable enough about what you are trying to create, you can speak to the sub-components that contribute to the desired effect. If you lack that language, it's very hard to guide it.

Capable people are able to execute at scale on an unprecedented level now. Less capable people will be able to very quickly ape familiar styles but it's harder to create something uniquely powerful without an understanding of and the language to articulate the elements that contribute to the desired creation.

As we continue to work with LLMs, it's crucial to recognize their role as skill magnifiers. They can supercharge productivity for the proficient and provide a solid foundation for novices. But their true potential lies in accelerating our learning and skill development.

The challenge lies in guiding these AI tools effectively, especially in creative realms. It requires not just technical know-how, but the ability to articulate nuanced, subjective qualities.

As creators and builders, our task is clear: leverage LLMs where they excel, but more importantly, develop the expertise and language to guide them effectively, especially in creative and subjective tasks. The future belongs to those who can seamlessly blend AI assistance with human creativity, insight, and the ability to provide nuanced guidance.

What's your take on this balance? How do you see it shaping the future of content creation?

The Critical Role of Human Expertise in AI-Assisted Creation