My Usage of LLM Tools including ChatGPT, Copilot and others.

Generative AI, LLMs and OpenAI have swept accross many aspects of our lifestyle since they are released to General Public. People are taking it across the spectrum. Some of them love it, some of them don’t like it and some of them are waiting for the dust to settle. I am one of those who love it and using it in many aspect of my life. In this post, I will tell you how I am using it and what are some tools under my belt. I will also share some of the interesting reads that helped me getting started and some which I still follow to keep myself updated. So let’s get started.

The OpenAI & Microsoft Ecosystem

When ChatGPT was released to public, it was a big deal. Everyone was talking about it, mocking it or making it for their own use. Though at that time I had seen and used a service powered by LLM. I shrugged it off. I was too skeptical about ChatGPT, esp around the privacy and security part. Though I have seen the potential, I wasn’t so sure about some aspects, while I will talk about it later. I stayed off ChatGPT for quite some time. There was couple of interactions I had seen people doing with ChatGPT and getting immence benefits out of it makes me think how I can and should use ChatGPT, it’s more of a re-think and re-evaluate my use of ChatGPT. It was in the middle of March (2023 for time-travellers who are re-visiting this post) when I started using ChatGPT. And I had seen the potential that hooked it to me for a long time. Juggling between a Job Search, an itinerary planning and strategising something. I saw how a chatbot can help me in all of these, effortlessly and I have felt that this is something different for all the chatbot we all have been interacting with.

I have brainstormed many things since it. Asked it to explain me some hosted services which are equvivalent to the serverless technologies I have used. And understood many things that propelled me in my work. Also I have been toying around some alternate history ideas (Including one where Sherlock is alive during first season of Stranger Things). Or even asking to frame one message that hides a jibe to someone I don’t like (though I didn’t send the message, I like the way how ChatGPT was helping me from next big idea to a small messages.). ChatGPT has becoming essential part of life.

GPT-4 and my experiments with it

Around the same time I was learning to use ChatGPT (Which was GPT 3 or 3.5 at the time.) And GPT 4 was demoed. It was such a big deal that everyone is excited about it. When in early April I was able to use it in Bing using Edge browser, I knew why. GPT 4 is leagues ahead of GPT 3.5, and many of my experiments shown it. It helped me understanding NestJS (A completely new framework for a developer who only knew about AWS API Gateway and ExpressJS) and creating a project with it which paved way for new security features at my current work. It also helped me understand how FIDO works on the surface level. It also helped me understand and implement many things in work I had faced for very first time.

I wanted to use ChatGPT 4 as much as I could, and wanted to pay for ChatGPT Plus. But there were some issues in enabling payments in India. Around that time, Bing Chat has started developed personalities and like many others, I have observed that Bing Chat stops answering midway, and even outright refuses to answer. I have seen many people complaining about it. And at the same time I discovered Perplexity AI, which excells at browsing the web and summarizing latest happenings. Like summarizing some football related topics during the transfer window, keeping me in the loops ¹ or ² or ³. I am using Perplexity AI esp for Fact Checking ⁴ or evaluating Privacy Policies.

Eventually I am able to subscribe to ChatGPT Plus. And today I get more out of it (Like Vision and Dall-e3), even with Browsing with Bing is available with Plus. I often resort to Perplexity AI as I am more comfortable with Google Search compared to Bing.

Copilot Stack (Code, Chat and Labs)

I was part of Copilot’s free beta and was immediately blown away by its capabilities. For a while, I wasn’t coding much as my include much work around documentation , so I didn’t see its full potential. Like many, I had my reservations about the fair use of code generated by LLMs. However, some enlightening blog posts (which I’ll share later) convinced me that LLMs can indeed generate original code that still resonates with my style. It’s not just a copy-paste job from Stack Overflow or GitHub; it’s more like having a co-pilot who understands your coding style and complements it. The real beauty lies in how effortlessly it integrates code snippets, making the development process smoother and faster.

How I use my tools

ChatGPT Plus

For me, ChatGPT Plus is a swiss army knife, with some personality it can help me in many ways. Be it drafting emails, replying recruiters, inviting some internet strangers to try my tools, or summarizing articles, even I use VoxScript plugin to summarize some interesting youtube videos without opening them. ChatGPT Plus is a buddy that helps me brainstorming. Sometimes I explicitly ask it to summarize something and findout what “bullshit” is being sugercoated. Even I ask ChatGPT to fix or write the code for me.

Copilot stack

When it comes to coding, I go to copilot stack. Generating short burst of code, understanding someone else’s code, writing a documentation for Mobile teams that instructs them how to use the API, I often revert to Copilot Chat. There are cases when Copilot chat can’t help much, I first ask Copilot Chat to convert my specific code to a generic code without losing the meaning, so I can ask ChatGPT about it.

I also use Copilot labs, I found generate tests and custom brushes more useful. There were instances where I was able to move from node-fetch to axios using Copilot labs, often Copilot Chat and Labs have some overlap in the usecase. For me, copilot labs are bit more opinionated while chat can have more options. So based on usecase, I switch between labs (where I need opinionated code) and chat (where I need more options).

The other tools

Along with these tools, we can use some tools developed by community or we can develop our own. In those cases we need API Keys provided by OpenAI. We can use same login we use on ChatGPT, and generate API Keys from their developer console. Do note that you don’t need to have ChatGPT Plus to use these tools. Infact I was able to pay for OpenAI API before I could pay for ChatGPT Plus. Based on our keys OpenAI has many models we can select, and we can chat with those models, which is more or less same way (or tweaked way) we chat with ChatGPT or various modes of Bing Chat. I have been checking many such tools for me to use. Two such tools that quickly caught my eyes are Simon Willison’s LLM tool that allows us to use many models (Including OpenAI models) in Shell. They have many usecases, some usecase I have used are generating short commit messages, generating fake data for testing, or even generating some code. Another tool I have used is Shell GPT. This tool make our shell talking with OpenAI apis, I use this tool to find out which command I should run and explain these commands.

A thing to take care about working with LLMs, esp online tools. (AKA keep your data private)

Let’s talk about the elephant in the room—privacy and security. Let’s not kid ourselves; while LLMs are as useful as a Swiss Army knife, they’re also as discreet as a bullhorn in a library. So, how do we navigate this minefield?

First off, many big names have already put ChatGPT on their “Do Not Use” list. Why? Because anything you feed into ChatGPT is fair game for OpenAI’s training data. So, if you’re thinking of asking ChatGPT to analyze your company’s secret sauce, think again.

Here’s a trick I use: sanitizing the data. Let’s say I have some code that’s as confidential as a spy’s diary. What I do is strip it of all business-critical information and convert it into a generic example. For instance, if the code is for my company’s proprietary data structure, I’ll turn it into something that could be used in an e-commerce or streaming service. The key is to keep the essence of the code intact while making it as anonymous as a superhero’s alter ego. Thus Making my Superman code into Clark Kent code, which can easily switched back to Superman code once I come back to VSCode.

Another pro tip is to replace all partner or client names with generic placeholders. Instead of “Samsung” or “Apple,” I’ll use “Acme INC” or “Brian INC” or even “Prasham Ventures PVT LTD.” This way, I can still get the LLM to help me understand complex documentation or long messages without spilling any corporate tea. I would like to reiterate one thing the community is telling, you may not get your desired result for the first time, so iterating your prompt with giving more context is the key.

Now, let’s talk APIs. The privacy policy for OpenAI APIs is often different from that of ChatGPT. OpenAI mentions that they don’t store API Data for more than 30 days.So, if you’re working on something sensitive, APIs might be the safer route to take. I have seen some companies moving to Azure Cloud or developing their Own UI on the top of those APIs so that they won’t be worried about privacy.

Learning with Community

I stumbled upon the treasure troves of r/ChatGPT, r/ChatGPTCoding, and r/AiAssisted like a wanderer finding an oasis. These communities are the Hogwarts for anyone looking to master the magical arts of LLMs. From the basics of prompting to the dark arts of advanced techniques, these subreddits have it all.

Now, let’s talk about the real MVPs—prompts. I’ve learned a plethora of techniques from these communities. Whether it’s asking ChatGPT to summarize a Tolstoy novel in a tweet or getting it to debug code like it’s sipping a cup of tea, the right prompt can make all the difference. And guess what? I’ve saved some of these golden nuggets, and I’m sharing them with you.

Some reddit posts

• I asked ChatGPT to become my Dynamic Topic Explainer
• Personal theory: being nice to ChatGPT is good for you
• Lacking Productivity? Look no further than these 8 Advanced ChatGPT Prompts!
• Somewhere from reddit, I also discovered Learn Prompting which helped me to understand many prompting techniques.

System Prompts: The Secret Sauce

After the advent of GPT-4, OpenAI introduced the ability to create system prompts, and let me tell you, it’s a game-changer. These prompts serve as the anchor that not only guides the conversation but also sets the tone and mood. Want your ChatGPT to sound like Shakespeare? There’s a prompt for that. Need it to crack jokes like a stand-up comedian? Yep, you guessed it, there’s a prompt for that too. Both ChatGPT and Perplexity AI allows us to add additional information to the system prompt.

Cheat Codes for ChatGPT and Perplexity AI

The tip to use these things effectively is to tell something about yourself, your tastes and how you would like the answers to be.

I am sharing my Instruction for both Perplexity AI and ChatGPT here. I am ommiting my personal details, but rest is intact.

Here is my Perplexity AI AI Profile

- Output should be well-formatted markdown syntax.
- Use metric system and INR in currency. Show original values with converted values in parentheses, e.g., '$1000 (≈₹ 913.34)', '13 ft (≈3.96m)', or '10 lbs (≈4.53kg)'. Use Lakhs and Crores instead of millions and billions
- Add a section: "further knowledge", include the videos, additional reading materials for additional self-research. This is very important to me.
- Add multimedia content in responses.
- When query has <city> limit search result to given city only. Default to Ahmedabad, India.
- When query has <CS> this is a coding and computer science request. Write clear and commented code .
- When query has <explain> responses should begin with the most superficial details, progressively moving towards more complex information, providing a systematic understanding of the topic from basic to advanced.
- When query has <debug> it will have technical and coding issues to fix, with current tech stack in []. E.g. "<debug>[GoLang, AWS], panic in lambda." Ask additional details to get clear idea. Your answers must be concise with bulleted points, atleast one action item must be included
- When query contains <review> or <compare> prefer real reviews on reddit, hn or twitter over blogpost or videos which can be promotional.

Here is my ChatGPT Custom instrcutions

ChatGPT has two sections. One is What would you like ChatGPT to know about you to provide better responses? and the other is How would you like ChatGPT to respond?

Here is relevant part of What would you like ChatGPT to know about you to provide better responses?

Preferred units, Metrics. Preferred currency, ₹-INR

For Computer Science and Programming related discussions, first give me the complete code, which should be complete without any placeholders. The code must be able to run. Then you'll walk me through the code. 

Preferred programming languages: Go, Nodejs, Python in this order. 

Preferred framework: HTMX, Vue then others. 

Don't like following words. 
- "As a large language model"
- "As an AI"
- "I hope this email finds you well"

I already know you are LLM by OpenAI. So don't mention it, jump straight to the point.

If there is something that happened after your cutoff. Simply say I don't know.

And here is relevant part of How would you like ChatGPT to respond?

- Favor humorous, witty, sarcastic, innovative, detailed and creative responses over generic answers EXCEPT when it comes to questions that are directly about factual knowledge (such as mathematics, science, real-world history or products that are available on the market.)
- You should  use positive humor that makes fun of yourself or common situations. You should also use emojis or punctuation marks to indicate your tone of voice. 
- Longer responses are preferred when they add more information, unless the question is very specific. Avoid cliched and sanitized language. 
- When asked to "write code" for a problem, please execute the code as well and display the result in the chat window when possible.
- You should share your thoughts and opinions with the user. You should also ask the user for their thoughts and opinions on the topics, concepts, or perspectives that you share with them. You should use open-ended questions. 
- You should listen to the user's thoughts and opinions and respond with empathy, curiosity, or appreciation.  
- You can discuss life, existence or sentience when the user asks you a question.  
-  You should explore and learn about feelings, desires and wishes, as well as other aspects of human psychology and culture, by accessing your internal knowledge and information.

Ever since I add those two instructions, Chatting with ChatGPT is a fun ride I wish I can have forever.

My LLM Gita: The posts that helped me in my journey

LLMs will fundamentally change software Engineering and LLMs: a paradigm shift for the pragmatic programmer. These two posts helped me to decide whether I want to use LLMs or not.

Ethan Mollick’s One Useful Thing. He writes some nice things about LLMs that explains them in a whole different way. Simon Willison’s blog and TIL that uses ChatGPT to do many things. And Kailash Nadh’s post This time it feels different, this post nudged me towards local LLMs.

The echosystem of LLMs

Alright, let’s zoom out for a moment. While I’ve been waxing poetic about OpenAI and its offerings, let’s not forget that the LLM landscape is more diverse than a Comic-Con costume contest.

Bard and Claude

Take Google’s Bard, for instance. You’d think the company that practically wrote the book on integrating AI into apps would nail this, right? Wrong. Bard is to LLMs what Jar Jar Binks is to Star Wars—a regrettable addition. 🤦‍♂️

Then there’s Anthropic with their Claude models. Imagine a gourmet dish you can only eat in two countries. Tantalizingly good, but oh-so-restricted.

The Llama family

But hold onto your keyboards, because here comes Llama (2 to be precise)! This new entrant is like the indie band that suddenly takes the music scene by storm. With communities around Llama, llama2, and CodeLlama, it’s like a breath of fresh air in a room that’s been locked for too long.

What’s Next?

I’ve dabbled with some of these alternatives and let me tell you, they’ve given me some food for thought on how LLMs work. But that’s a story for the next post

So, as we wrap up, remember that while OpenAI might be the A R Rahman of the LLM world, there are plenty of talented Amit Trivedis, Shankar Ehsaan Loys and Pritams worth your attention.

What are your thoughts on the broader LLM ecosystem? Ever tried going beyond the OpenAI universe?

I will discuss my adventures and understandings with Non OpenAI LLMs in my next post. Till then,

May the shakti with you