Package edu.tufts.hrilab.llm.llama

Class LlamaClient

java.lang.Object
edu.tufts.hrilab.llm.llama.LlamaClient
public class LlamaClient extends Object

  • Constructor Details

    • LlamaClient

      public LlamaClient()

  • Method Details

    • setEndpoint

      public void setEndpoint(String endpoint)
      Sets the endpoint for the llama.cpp server.
      Parameters:
      endpoint - - The base endpoint to set

    • setStopWords

      public void setStopWords(String[] stopWordsArr)
      Sets the stop words array for terminating token stream on matching words.
      Parameters:
      stopWordsArr - - An array of stop words to set

    • setTemperature

      public void setTemperature(float temperatureFloat)
      Sets the temperature of the model.
      Parameters:
      temperatureFloat - - The temperature value to set

    • completion

      public LlamaCompletionResponse completion(Prompt prompt)
      Generates a completion of the given prompt using the llama.cpp API service.
      Parameters:
      prompt - the prompt for which to generate the completion
      Returns:
      a LlamaCompletionResponse containing the generated completion and additional information

    • completion

      public LlamaCompletionResponse completion(Symbol prompt)

    • completion

      public LlamaCompletionResponse completion(Symbol model, Symbol prompt)
      Generates a completion of the given prompt and model using the llama.cpp API service.
      Parameters:
      prompt - the prompt for which to generate the completion
      Returns:
      a LlamaCompletionResponse containing the generated completion and additional information

    • completion

      public LlamaCompletionResponse completion(String prompt)

    • chatCompletion

      public LlamaCompletionResponse chatCompletion(Symbol prompt)
      Generates a chat completion using the LLaMA service and a prompt string.
      Parameters:
      prompt - the prompt string provided by the user
      Returns:
      the LlamaCompletionResponse containing the completed chat.

    • chatCompletion

      public LlamaCompletionResponse chatCompletion(String prompt)

    • chatCompletion

      public LlamaCompletionResponse chatCompletion(List<Message> messages)
      Performs an LLaMA chat completion using the specified model and the given list of messages. This method creates a new LlamaRequestBody object with the provided model and messages.
      Parameters:
      messages - a list of messages to be sent for chat completion
      Returns:
      an LlamaCompletionResponse object representing the response from LLaMA

    • chatCompletion

      public LlamaCompletionResponse chatCompletion(Chat chat)
      Performs a chat completion using LLaMA with the set model and the given chat messages.
      Parameters:
      chat - the chat containing the messages to use for the completion
      Returns:
      an LlamaCompletionResponse object representing the response from LLaMA