# Overview No Code Generative AI The success of Generative AI hinges on methods such as fine-tuning and in-context learning. Ensuring its adaptability to specific domains allows a broader range of individuals and businesses to benefit without deep technical expertise. ## **What's Texti.ai about?** This simple mindmap illustrates how a response (or completion) to the prompt "*Tell me about the drug Aspirin?*" can be altered when the foundational AI model is customized using In-Context Learning or Fine-tuning.
It's worth noting that **In-Context Learning** and **Fine-tuning** often necessitate substantial coding, data preparation, and domain-specific knowledge in data science. However, given the current scarcity of AI talent, there's an emerging need for tools that empower individuals to harness the capabilities of AI models without the necessity of coding.\ \ [Texti.ai](https://www.texti.ai/), recognizing this challenge, offers businesses and individuals the ability to delve into both fine-tuning and in-context learning—two key pillars of AI customization—without writing a single line of code. Simply upload your dataset, define the desired task, and Texti.ai handles the intricate processes behind the scenes. # Task A "task" refers to a specific natural language processing task that can be performed by a language model. By fine-tuning your language model for a particular task, you can improve its accuracy and make it better suited to your specific needs. Texti.ai supports the following task categories that you can fine-tune your language models for: * Text Classification (Fine-tuning) * Text Generation (Fine-tuning) * AI Co-Pilot (In-Context Learning) # Text Classification Text classification involves assigning predefined categories or labels to text documents. Here are some real-life business use cases for text classification: 1. ***Sentiment Analysis***: Classify customer feedback to identify areas for improvement in product or service. 2. ***Fraud Detection***: Classify the transaction as fraudulent or not based on other features of the text, without explicitly identifying specific entities. 3. ***Topic Categorization***: Automatically sort customer support tickets based on topic to improve response time and efficiency. 4. ***News Article Categorization***: Categorize news articles to monitor trends and stay up-to-date with industry developments. 5. ***Email Filtering***: Identify spam emails to improve email filtering and security. 6. ***Resume Screening***: Classify job resumes to streamline the recruitment process. check how to prepare data for Text Classification task. # Text Generation Text generation involves generating natural language text, such as product descriptions, ad copy, and more. Here are some real-life business use cases for text generation: 1. ***Product Description***: Generate natural language text that accurately describes a product or service. 2. ***Ad Copy***: Generate ad copy that effectively communicates the benefits and features of a product or service. 3. ***Chatbot***: Build a chatbot to assist customers with product information and support. 4. ***FAQ Automation***: Automatically answer frequently asked questions to reduce customer support workload. 5. ***Virtual Assistant***: Enable virtual assistants to provide personalized recommendations and advice. # AI Co-Pilot # Data & Labeling To fine-tune, you will need to provide it with relevant training data. This training data should be in a specific format that includes both a "prompt" and a "completion". Texti.ai makes it easy to prepare and upload your data. Connectors for uploading your prompt and completion dataset include: CSV, Google Sheets, PostgreSQL, and MS SQL Server. Once your data is uploaded, you can edit or add labels to your dataset directly on our user interface, depending on the task that you have chosen. Here's a breakdown of prompt and completion formats for different task categories: 1. [Text Classification](/data-labeling/text-classification) 2. [Text Generation ](/data-labeling/text-generation)
# Text Classification For text classification tasks, you can upload a dataset with labeled examples of text, where each example is labeled with a specific category. The categories can be anything that you want to classify your text into, such as sentiment (positive, negative, neutral), topics, or any other classification that is relevant to your use case. Here is an example of what your dataset might look like: | Prompt | Completion | | ----------------------------------------------------- | ---------- | | This restaurant has great food and excellent service. | positive | | The package arrived late and in poor condition. | negative | | I love this movie, it's one of my all-time favorites. | positive | | The customer support was unhelpful and rude. | negative | | The weather is beautiful today. | neutral | # Text Generation For text generation tasks, you can upload a dataset with prompts and expected completions. Each prompt should provide context and cues for the model to generate a coherent and relevant completion. Here is an example of what your dataset might look like:
| Prompt | Completion | | ------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | Name: Women's T-Shirt ; Material: Cotton ; Color: Blue ; Size: S ; Price: $19 | A soft and comfortable women's t-shirt made from high-quality cotton material, featuring a classic blue color that's perfect for everyday wear. Available in size S. | | Name: Men's Hoodie ; Material: Fleece ; Color: Gray ; Size: M ; Price: $29 | A warm and cozy men's hoodie made from premium fleece material, featuring a stylish gray color that's perfect for casual wear. Available in size M. | | Name: Women's Dress ; Material: Silk ; Color: Red ; Size: L ; Price: $99 | A luxurious and elegant women's dress made from high-quality silk material, featuring a stunning red color that's sure to turn heads. Available in size L. | | Name: Men's Sneakers ; Material: Leather ; Color: Black ; Size: 9 ; Price: $49 | A stylish and comfortable pair of men's sneakers made from premium leather material, featuring a sleek black color that's perfect for any occasion. Available in size 9. | | Name: Women's Jeans ; Material: Denim ; Color: Indigo ; Size: 27 ; Price: $39 | A classic pair of women's jeans made from durable denim material, featuring a versatile indigo color that's perfect for everyday wear. Available in size 27. | # AI Co-Pilot # Fine-tuning ## What is Fine-tuning? Fine-tuning is a powerful technique used to improve the accuracy of AI models. It involves training a pre-existing model on a smaller, more specific dataset to learn a new task. This allows the model to make more accurate predictions for the specific task at hand. [Please refer to this mindmap to understand how the responses of foundational models are altered based on the chosen customization technique.](/) ## Fine-tuning vs In-Context Learning vs Prompt Engineering **Fine-tuning** involves adapting a pre-trained language model by training it on a smaller, domain-specific dataset. While this method is effective for domain adaptation and enhancing accuracy, its most distinguishing advantage is providing structure to the model's responses or completions. However, achieving optimal results through fine-tuning requires a substantial amount of data, computing power, and technical expertise, making the process potentially time-consuming.\ \ **In-Context Learning (for Building AI Co-Pilots)** using the Retrieval Augmented Generation (RAG) pattern enhances large language models (LLMs) by integrating them with external information retrieval systems. Rather than relying exclusively on an LLM's pre-trained knowledge, RAG taps into external sources to provide contextually rich responses. This is crucial since the knowledge within LLMs is confined to a specific cutoff date. Building AI Co-Pilots utilizing this approach demands a considerable amount of technical expertise, given the requirements such as creating embeddings, devising chunking strategies, and establishing a vector database. **Prompt engineering** involves crafting a prompt or a series of prompts that guide the language model to produce specific outputs. This method can yield high-quality results with minimal data and computing power. Unlike fine-tuning, prompt engineering is more accessible; it doesn't demand extensive technical knowledge and allows users to utilize any model without the need for customization. # Model Selection You can currently fine-tune the following models on Texti : 1. **davinci-002** - \ Davinci is the most capable model, excelling in deep understanding of content, complex intent, cause and effect, and summarization for specific audiences. It requires more compute resources and is slower compared to other models. 2. **babbage-002** - \ Babbage is suitable for straightforward tasks like simple classification. It performs well in semantic search, ranking how well documents match search queries. 3. **LlaMa-65B** (limited availability) - \ LlaMa-65B is a more efficient model, part of a series emphasizing reduced compute demands. These models facilitate versatile fine-tuning and are trained on vast data sets, including texts from the top 20 spoken languages 4. **BERT -** \ BERT revolutionized NLP, capturing context in both text directions. It's adaptable across domains but demands more data for fine-tuning.
> *While cost and speed are important considerations, the availability of data can also significantly influence the decision-making process when selecting the appropriate model.* ### Data Availability & Model Choice If you have access to a large amount of data, you can effectively fine-tune a less powerful, less expensive OpenAI base model and achieve satisfactory results. However, in cases where you have limited data, leveraging a more powerful and expensive OpenAI base model can be advantageous.
# Hyperparameters Hyperparameters are the settings that determine how the model learns from the data, such as the learning rate, the batch size, and the number of epochs. These settings can greatly impact the model's performance and accuracy, so it's important to choose them carefully. Experienced data scientists often experiment with different hyperparameters and evaluate the accuracy of their predictions using metrics like training loss, validation loss, and other evaluation metrics. They use these metrics to determine the performance of the model and find the best possible hyperparameters for their specific task. **For example**, if a data scientist is working on a text classification task, they might experiment with different values for the learning rate and batch size, and then evaluate the performance of the model using metrics like precision, recall, and F1-score. By repeating this process with different hyperparameters, they can fine-tune the model to achieve the best possible performance on their specific task. In the context of Texti.ai, users can choose their hyperparameters using the one-click fine-tune option or manually input hyperparameters that they have found to be optimal for their specific task. * **One-click fine-tune**: This option follows the recommended hyperparameters for the pre-trained model. * **Advanced setting** (Manual hyperparameter optimization): Advanced users can manually enter the hyperparameters they want to use for fine-tuning. Note that this section is aimed at users who are not familiar with hyperparameters. Advanced users can skip this section and manually enter the hyperparameters they want to use. These settings can greatly impact the model's performance and accuracy, so it's important to choose them carefully. Here are the explanations of the hyperparameters: 1. **n\_epochs**: The number of times the model should see the entire dataset during training. This is also known as the number of training iterations. A higher number of epochs can lead to better performance, but can also increase the risk of overfitting.\ Recommended values: - 2-5 for quick experimentation - 10-50 for fine-tuning. 2. **Test Set** (%): This hyperparameter specifies the percentage of the entire dataset to be reserved for testing. When splitting data for model training and evaluation, this value determines the proportion allocated to the test set. The test set is crucial as it's used to evaluate the performance of the model after fine-tuning, ensuring it generalizes well to unseen data. Typically, a certain percentage (e.g., 20-30%) might be set aside for this purpose, though the exact value can vary based on the dataset and specific use case. 3. **batch\_size**: The number of examples that are processed in one forward/backward pass during training. A smaller batch size can lead to a slower convergence but can also result in better performance. Recommended values: - 4-16 for quick experimentation - 32-64 for fine-tuning. 4. **learning\_rate\_multiplier**: A multiplier that affects the learning rate of the model during training. A smaller learning rate can lead to slower convergence but can also help prevent the model from overshooting the optimal solution.\ Recommended values: - 0.01 for quick experimentation - 0.1-1 for fine-tuning. 5. **prompt\_loss\_weight**: A weighting factor that determines how much importance should be given to the prompt text during training. For example, if the prompt\_loss\_weight is set to 0.8, the model will focus more on generating text that matches the prompt text rather than generating text that is completely novel. This hyperparameter is only valid for text generation tasks.\ Recommended values: - 0.5-0.9 for quick experimentation - 0.8-1 for fine-tuning. 6. **compute\_classification\_metrics**: A flag that determines whether to compute classification metrics such as precision, recall, and F1 score during training. This hyperparameter is only valid for text classification tasks.\ Recommended values- True. 7. **classification\_n\_classes**: The number of classes to classify the text into. This hyperparameter is only valid for text classification tasks.\ Recommended values: - 2 for binary classification - 3 or more for multi-class classification. 8. **classification\_positive\_class**: The name or index of the positive class in the classification task. This hyperparameter is only valid for text classification tasks.\ Recommended values: - "positive" for binary classification - the index of the positive class for multi-class classification. 9. **classification\_betas**: The beta value(s) to use for F-beta score computation. This hyperparameter is only valid for text classification tasks.\ Recommended values: - 0.5 for F1 score - 1 for precision/recall balance. # Model Fine-tuning Once the hyperparameters are set for fine-tuning, Texti.ai handles all necessary steps automatically, with a streamlined process that includes data preparation, training the model, and displaying evaluation metrics of the fine-tuned model. Here's a breakdown of what happens during these steps: #### **Data Preparation** This step involves preparing your data for fine-tuning, including splitting it into training and validation sets and uploading the files to Open AI's servers. 1. Data splitting: In this step, the uploaded data is split into separate training and validation sets with an 80-20% split when not set manually to ensure that the model is trained on a representative sample of data. For classification tasks, the training and test dataset are automatically balanced, and the category that is finally chosen is the one that has the maximum votes. Read more on [Automatic Dataset Balancing For Classification Tasks.](/automatic-dataset-balancing-for-classification-tasks) 2. Training file upload: As the name suggests, this is where the training data is uploaded to model providers (Open AI) or our servers if you choose open-source models. 3. Validation file upload: Validation data is uploaded to AI model providers servers. #### **Model Fine-tuning** This step involves actually fine-tuning the base selected model for the tasks selected. 1. Model training: In this step, the platform trains the model using the data that you have provided, using the hyperparameters that you have selected in the previous step. 2. Deployment: Once the model is trained, it is deployed to the platform so that you can begin using it for inference. #### **Model Evaluation** This step involves evaluating the performance of your model using various metrics. * Evaluation metrics: The platform provides several metrics to help you evaluate the performance of your fine-tuned model, including accuracy, precision, recall, and F1 score. These metrics can be used to determine the effectiveness of your model for the task at hand. # Model Evaluation Once the fine-tuning is complete, you can use the fine-tuned model to generate text for your specific use case. The inference page allows you to input the prompt text and get the model's generated completion text as output. #### Evaluation Metrics During fine-tuning, the model's performance is evaluated using different evaluation metrics. Some of the commonly used evaluation metrics are: * **Training Loss:** The training loss is the value that the model is trying to minimize during training. It measures how well the model is able to fit the training data. A low training loss can indicate that the model is learning the patterns in the training data well, but it can also indicate overfitting if the validation loss is high. * **Validation Loss:** The validation loss is the value that measures how well the model is generalizing to new data that it has not seen before. A low validation loss indicates that the model is able to generalize well to new data. If the validation loss is much higher than the training loss, it indicates that the model may be overfitting to the training data. In addition to training loss and validation loss, there are several other evaluation metrics that can be used to evaluate the performance of a fine-tuned model. For text classification tasks, the following metrics are commonly used: * **Precision:** The fraction of true positive predictions out of all positive predictions. In other words, precision measures how many of the predicted positive results are actually positive. * **Recall:** The fraction of true positive predictions out of all actual positive examples. In other words, recall measures how many of the actual positive results were correctly predicted. * **F1 score:** A harmonic mean of precision and recall. It's a balanced measure that combines the two metrics to give an overall evaluation of the model's performance. * **Accuracy:** The fraction of correct predictions out of all predictions. In other words, accuracy measures how many of the predictions made by the model are correct. # In-Context Learning ### What is In-Context Learning? In-Context Learning (for Building AI Co-Pilots) using the Retrieval Augmented Generation (RAG) pattern enhances large language models (LLMs) by integrating them with external information retrieval systems. \ Rather than relying exclusively on an LLM's pre-trained knowledge, RAG taps into external sources to provide contextually rich responses. This is crucial since the knowledge within LLMs is confined to a specific cutoff date. Building AI Co-Pilots utilizing this approach demands a considerable amount of technical expertise, given the requirements such as creating embeddings, devising chunking strategies, and establishing a vector database.\ \ [Please refer to this mindmap to understand how the responses of foundational models are altered based on the chosen customization technique.](/) # Model Selection You can currently use following models to build AI Co-Pilot using In-Context Learning : 1. **GPT-3.5-turbo (16K Token Size)** - The best model in the GPT-3.5 series from Open.
2. **GPT-4 (32K Token Size)** - The most capable GPT model series to date. Able to do complex tasks, but slower at giving answers.
3. **LlaMa-65B (8K Token Size)** -\ This model currently has limited availability. \ LlaMa-65B is a more efficient model, part of a series emphasizing reduced compute demands. # Hyperparameters #### Embeddings: In the context of Large Language Models, embeddings are mathematical representations of words in a high-dimensional space. They capture the semantic relationships between words and serve as a foundation for understanding and generating text. Currently, only "text-embeddings-ada-002" is supported by Texti, but many domain-specific open-source models will soon be available. #### Similarity Measure: The similarity measure defines how the model gauges the closeness or similarity between different pieces of text. Currently, Texti only supports "Cosine" as a similarity measure. #### Chunk Size (Tokens): This refers to the number of tokens (words or subwords) processed together as a single block or 'chunk' when documents are converted into embeddings. # Evaluation This feature is currently not available on Texti.ai, but it will be available soon. # AI Agent AI agent developments have not yet been integrated into Texti's platform. However, Texti is currently conducting pilot programs with selected enterprises to automate any business process they may have. These AI agents are poised to elevate enterprise automation to new levels, ranging from closing feedback loops in mere minutes to enabling algorithmic trading agents. They are set to be the driving force behind the top businesses of the future. If you are interested in participating in this pilot program, please [contact us](mailto:undefined).
Explore our use cases here: * [Real-Time Feedback-to-Action AI Agent](/ai-agent/real-time-feedback-to-action-ai-agent) # Real-Time Feedback-to-Action AI Agent John, an eCommerce Manager, is seeking to enhance his business processes through the use of generative AI models. Specifically, he aims to automate the feedback loop, ensuring that any negative customer feedback received on his webshop is addressed within a minute. \ \ With the help of Real-Time Feedback-to-Action AI Agent developed by [Texti.ai](https://www.texti.ai/), John can successfully close this feedback loop instantaneously. Previously, there was a BI dashboard that provided insights, but unfortunately, these feedbacks never became actionable items, leading to unhappy customer experiences and increasing the churn rate.

Texti's AI Agent

The AI Agent shown in the image above involves the following steps: 1. The AI Agent reads customer feedback submitted to the webshop when it is called using a REST API connector to this agent running independently. 2. The prompt runs through following fine-tuned, customized models trained on the business-specific training data - * [x] The sentiment classification model processes the feedback to determine its sentiment. If the sentiment of the incoming feedback is 'Negative', then it is passed on to the next model. If the sentiment is not negative, the process stops for that piece of feedback. * [x] The feedback classification model receives the feedback with negative sentiment and performs multi-label classification to identify specific issues, categorizing the feedback into topics such as 'Delivery Delay' and 'Packaging Issue'. 3. Once the negative sentiment and specific issues are identified, the AI Agent performs two actions simultaneously: * [x] Sends an email to John's team, alerting them of the customer issue and urging immediate action. * [x] Updates the company's database with the categorized feedback for record-keeping and further analysis. As the AI agent runs through all the steps, Texti.ai's AI Agent comes with a log viewer that helps users to build, run the AI agent, and check logs/actions in real-time. The above AI agent, after execution, generates detailed logs for each node process in the workflow.

Texti's AI Agent logs

This AI Agent runs independently on John's existing [Azure Cloud](https://azure.microsoft.com/en-us/free/cloud-services/search/) infrastructure, ensuring hassle-free integration with existing resources.\ \ [Read more about Texti's AI Agent pilot program for enterprises.](/ai-agent) # How to Get an OpenAI API Key & Organization ID 1. Go to [OpenAI's Platform](https://platform.openai.com/) website and create an account (or if you already have an account [click here](https://platform.openai.com/account/api-keys)). Once you’ve created an account or have logged into an existing account, you’ll see your name and your profile icon at the top-right corner of OpenAI’s platform homepage. To get an API Key, click on your name in the top-right corner to bring up a dropdown menu. Then, click the “[View API keys](https://platform.openai.com/account/api-keys)” option. Select "Create new secret key" to generate an API key. Add the generated key to your [Texti](https://api.texti.ai/account) account.\
2. To retrieve the organization ID, click on "settings" and locate it as shown in the image below. Add this ID to your Texti account. Now you can fine-tune GPT-3 without coding!\ \
# Automatic Dataset Balancing For Classification Tasks The dataset used for fine-tuning a model plays a vital role in achieving accurate classification results. In scenarios where the dataset is imbalanced, meaning that the number of samples in each class varies significantly, the fine-tuned model may become biased towards the majority class, leading to suboptimal performance for the minority classes. {% hint style="info" %} Texti.ai incorporates an automatic dataset balancing mechanism during the fine-tuning process for classification tasks, which helps address this problem. {% endhint %} To address this problem, Texti.ai automatically balances your dataset when you fine-tune it for classification tasks. By doing so, the model can be trained on a representative set of data that contains an equal proportion of samples from each class, enhancing its ability to classify instances accurately across all classes. Let's take an example utilizing this [dataset](https://docs.google.com/spreadsheets/d/1_0Wc1a9pZRJUTVRlcaei86xXiAHvSHwRycIHwPC8a9o/). The dataset consists of 8,001 pairs of prompts and completions, with each prompt assigned to one of three classes: Positive, Negative, or Neutral. The original distribution of instances is as follows: * Negative: 2674 instances * Positive: 2727 instances * Neutral: 2600 instances In this scenario, the "Neutral" class has the lowest number of instances, specifically 2600 samples. Consequently, for the fine-tuning process, the final dataset (training & validation) will be adjusted to include a total of 7800 instances, with an equal distribution of 2600 samples from each class.