Package edu.tufts.hrilab.llm.hf

Class HFClient

java.lang.Object

edu.tufts.hrilab.llm.hf.HFClient

public class HFClient
extends Object

Field Summary

Fields

Modifier and Type	Field	Description
String	model
String	service

Constructor Summary

Constructors

Constructor	Description
HFClient()

Method Summary

Modifier and Type	Method	Description
LlamaHFChatCompletionResponse	llamaHFChatCompletion(Symbol userPrompt)	Generates a chat completion using the LLaMA HuggingFace Inference Endpoint and a prompt string.
LlamaHFChatCompletionResponse	llamaHFChatCompletion(Symbol userPrompt, Symbol systemPrompt)
LlamaHFChatCompletionResponse	llamaHFChatCompletion(Symbol userPrompt, Symbol systemPrompt, String model)
LlamaHFChatCompletionResponse	llamaHFChatCompletion(Chat chat)	Performs a chat completion using LLaMA with the set model and the given chat messages.
LlamaHFChatCompletionResponse	llamaHFChatCompletion(Chat chat, Symbol model)
LlamaHFChatCompletionResponse	llamaHFChatCompletion(Chat chat, String model)
LlamaHFChatCompletionResponse	llamaHFChatCompletion(LlamaHFCompletionRequestBody requestBody)
LlamaHFChatCompletionResponse	llamaHFChatCompletion(LlamaHFCompletionRequestBody requestBody, String model)
LlamaHFChatCompletionResponse	llamaHFChatCompletion(String userPrompt)
LlamaHFChatCompletionResponse	llamaHFChatCompletion(String userPrompt, String systemPrompt)
LlamaHFChatCompletionResponse	llamaHFChatCompletion(String userPrompt, String systemPrompt, String model)
LlamaHFChatCompletionResponse	llamaHFChatCompletion(List<Message> messages)
void	setMaxNewTokens(int maxNewTokens)	Sets the max number of tokens included in the response (Not including the provided prompt)
void	setModel(String serviceStr, String modelStr)
void	setTemperature(float temperatureFloat)	Sets the temperature of the model.
T5BaseCompletionResponse	t5BaseCompletion(T5BaseCompletionRequestBody requestBody)
T5BaseCompletionResponse	t5BaseCompletion(T5BaseCompletionRequestBody requestBody, String model)	Sends a Llama completion request with the given model and request body, and returns the completion response.
T5BaseCompletionResponse	t5BaseCompletion(String prompt)
T5BaseCompletionResponse	t5BaseCompletion(String prompt, String model)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

service

public String service

model

public String model

Constructor Details

HFClient

public HFClient()

Method Details

setModel

public void setModel(String serviceStr,
 String modelStr)

t5BaseCompletion

public T5BaseCompletionResponse t5BaseCompletion(String prompt)

t5BaseCompletion

public T5BaseCompletionResponse t5BaseCompletion(String prompt,
 String model)

t5BaseCompletion

public T5BaseCompletionResponse t5BaseCompletion(T5BaseCompletionRequestBody requestBody)

t5BaseCompletion

public T5BaseCompletionResponse t5BaseCompletion(T5BaseCompletionRequestBody requestBody,
 String model)

Sends a Llama completion request with the given model and request body, and returns the completion response. https://github.com/ggerganov/llama.cpp/tree/master/examples/server

Parameters:

requestBody - the request body containing the prompt and other parameters for the completion request

Returns:

the completion response from the TextSynth API

setTemperature

public void setTemperature(float temperatureFloat)

Sets the temperature of the model.

Parameters:

temperatureFloat - - The temperature value to set

setMaxNewTokens

public void setMaxNewTokens(int maxNewTokens)

Sets the max number of tokens included in the response (Not including the provided prompt)

Parameters:

maxNewTokens - - The number of tokens to set

llamaHFChatCompletion

public LlamaHFChatCompletionResponse llamaHFChatCompletion(Symbol userPrompt)

Generates a chat completion using the LLaMA HuggingFace Inference Endpoint and a prompt string.

Parameters:

userPrompt - the prompt string provided by the user

Returns:

the LlamaHFChatCompletionResponse containing the completed chat.

llamaHFChatCompletion

public LlamaHFChatCompletionResponse llamaHFChatCompletion(Symbol userPrompt,
 Symbol systemPrompt)

llamaHFChatCompletion

public LlamaHFChatCompletionResponse llamaHFChatCompletion(Symbol userPrompt,
 Symbol systemPrompt,
 String model)

llamaHFChatCompletion

public LlamaHFChatCompletionResponse llamaHFChatCompletion(String userPrompt)

llamaHFChatCompletion

public LlamaHFChatCompletionResponse llamaHFChatCompletion(String userPrompt,
 String systemPrompt)

llamaHFChatCompletion

public LlamaHFChatCompletionResponse llamaHFChatCompletion(String userPrompt,
 String systemPrompt,
 String model)

llamaHFChatCompletion

public LlamaHFChatCompletionResponse llamaHFChatCompletion(List<Message> messages)

llamaHFChatCompletion

public LlamaHFChatCompletionResponse llamaHFChatCompletion(Chat chat)

Performs a chat completion using LLaMA with the set model and the given chat messages.

Parameters:

chat - the chat containing the messages to use for the completion

Returns:

an LlamaHFChatCompletionResponse object representing the response from LLaMA

llamaHFChatCompletion

public LlamaHFChatCompletionResponse llamaHFChatCompletion(Chat chat,
 Symbol model)

llamaHFChatCompletion

public LlamaHFChatCompletionResponse llamaHFChatCompletion(Chat chat,
 String model)

llamaHFChatCompletion

public LlamaHFChatCompletionResponse llamaHFChatCompletion(LlamaHFCompletionRequestBody requestBody)

llamaHFChatCompletion

public LlamaHFChatCompletionResponse llamaHFChatCompletion(LlamaHFCompletionRequestBody requestBody,
 String model)