Have You Been Using Large Language Models Wrong This Whole Time?

Large Language Models (LLMs) like ChatGPT, Google Gemini, and Anthropic Claude are incredibly powerful tools, but most people only scratch the surface of what they can do. To truly get the most out of them, you need to do more than just ask questions—you need to engage, iterate, and make the models work for you. Let’s uncover how you might be using LLMs wrong, and how to turn things around with some game-changing strategies.

1. Crafting Prompts That Actually Get Results

The way you talk to an LLM can make all the difference. Think of prompts as giving directions—if you’re too vague, the model gets lost; if you’re too rigid, you miss out on creative possibilities.

  • Be Clear, but Allow Exploration: A good prompt directs the model but leaves room for creativity. Instead of just saying, “Explain photosynthesis,” try, “Explain photosynthesis in a way that would inspire a 12-year-old to love science.” This way, you’re giving it a chance to be both informative and engaging.
  • Stack Your Prompts for Maximum Insight: Don’t just stop at one question. Start broad and refine. For example: “Give me an overview of sustainable energy technologies.” Then, follow with, “Which of these technologies is best for urban areas and why?” This builds depth and helps break complex ideas into bite-sized pieces.
  • Use Each Model’s Unique Strengths: Each LLM has its specialties. ChatGPT shines with conversational responses, while Google Gemini is ideal for technical precision. Know your tools and tailor your prompts accordingly.

Walkthrough: Imagine you’re planning a community garden. You could start by asking ChatGPT, “What are some good crops for a community garden in a temperate climate?” Once you have the list, follow up with, “How can we organize these crops to promote natural pest control?” Then make it actionable by asking, “Create a month-by-month gardening guide for these crops.” This kind of layering turns vague ideas into a full-blown plan.

2. Building Context Like a Real Conversation

LLMs aren’t magic genies—they work best when you give them the right context and treat the interaction like a meaningful conversation.

  • Set the Scene: Before diving in, give the model some background. For example, say, “Imagine we’re designing a small garden and need it to be sustainable and visually appealing.” This helps set the stage for richer, more useful answers.
  • Use Context to Build Depth: Instead of repeating yourself, refer back to what’s already been discussed: “Earlier, you suggested rainwater collection—how can we make that low maintenance?” This continuity makes interactions smoother and more insightful.
  • Iterate, Iterate, Iterate: Don’t expect the perfect answer on the first try. Think of the LLM as your brainstorming partner. Start broad, then refine by asking, “What are the risks of implementing this idea?” to get a more complete picture.

Walkthrough: Suppose you’re working on a remote work policy. Start with, “What are the main components of an effective remote work policy?” Then ask, “How can we address employee burnout in this policy?” Finally, follow up with, “What’s the best way to communicate these policies to ensure buy-in?” Layering questions adds depth and ensures nothing is missed.

3. Follow-Ups That Make a Difference

The first answer is often just the beginning. Effective follow-ups can take a generic response and turn it into something truly valuable.

  • Dig Deeper: Push past the surface level. If the model suggests a strategy for productivity, ask, “What are the psychological benefits of this strategy?” or “How does it address common challenges in remote work?”
  • Switch Perspectives: Get a 360-degree view by switching perspectives. Ask, “How would a skeptic see this solution?” or “How would this work with limited resources?” These kinds of questions help you see blind spots.
  • Make It Actionable: Vague ideas are common—turn them into something actionable by asking, “What are the first three steps someone should take to implement this idea?” This ensures that insights are ready for real-world use.

Walkthrough: Let’s say you’re trying to improve customer service. Start with, “What are some key strategies to improve response times?” Then ask, “What challenges might a small business face when implementing these strategies?” Finally, push for specifics: “How can we measure success over the next six months?” This turns abstract advice into a step-by-step plan.

4. Structuring and Summarizing That Actually Helps

LLMs can give you a lot of information, but turning it into something useful is up to you.

  • Summarize with a Purpose: Ask for summaries with intent. Instead of just “Summarize this,” say, “Summarize this for an executive who only has 30 seconds.” Or, “Turn this into bullet points for a kickoff meeting.” The more specific you are, the better the outcome.
  • Create Tools, Not Just Text: Don’t stop at answers—ask for actionable formats. For example, “Turn this information into a roadmap for a marketing campaign.” This helps you get something directly usable rather than just a wall of text.
  • Simplify for Clarity: If the response is too complex, don’t hesitate to ask, “Explain this like I’m a high schooler,” or “Make this more concise.” It’s your job to get clarity.

Walkthrough: Once you’ve got the information you need, ask, “Summarize this as an actionable guide for beginners.” Then refine it: “Create a checklist from this guide that someone could use to track their progress.” This takes abstract information and makes it practical.

5. Choosing the Right LLM for the Job

Not all LLMs are created equal. Understanding what each model does best can help you get the right answer every time.

  • ChatGPT (OpenAI): Great for conversational flow, creative brainstorming, and discussions that benefit from empathy and nuance.
  • Google Gemini: Ideal for detailed technical analysis and integrating structured data. Perfect for research-heavy or accuracy-focused tasks.
  • Anthropic Claude: Focuses on safety and ethics, making it perfect for handling sensitive topics or simplifying complex concepts with care.

Walkthrough: Need to tackle an ethical issue? Start with Claude: “What ethical concerns should we consider when implementing AI in hiring?” Once you have that list, switch to ChatGPT for, “Frame these concerns in a conversational style for a presentation.” Using multiple models can make your workflow more effective by playing to each of their strengths.

Final Thoughts

If you’ve been using LLMs just to ask basic questions, you’re missing out on their real power. It’s about creating a strategy—layering prompts, building meaningful context, asking the right follow-ups, and structuring the output to be immediately useful. By treating interactions like a true conversation and pushing for depth, you can unlock the true potential of these tools.

Next time you use an LLM, remember: challenge it, iterate, and dig deeper. The more you put into the interaction, the more you’ll get out of it. Don’t settle for surface-level—go beyond, and you’ll be surprised at the results.