工具_羊驼是支持开源 LLM 使用的工具。
介绍
tool_llama是一个专门为与llama-cpp-python服务无缝集成而设计的辅助工具包。当前版本的服务为[V0.2.26]，使用其功能工具时，遵循特定的使用方法非常重要，如下所述：
python3 -m llama_cpp.server — model
这意味着采用较不知名的functionary-7b-v1模型，并在非工具讨论中牺牲chat_format。在llama-cpp-python中，chat_format被设置为'functionary'，但尚未实现与其他模型的chat_format兼容。理想情况下，我们应该能够在普通的聊天中使用工具，类似于OpenAI的方法。现在，大多数现有的开源模型在使用提示时可以有效地利用工具。因此，开发了tool_llama来增强当前开源工具链的功能。它拥有以下值得注意的特点：
1. 提供与Pydantic样式工具输入格式的兼容性 2. 在外部，以装饰器方式使工具和数据模型的引入变得更加便捷 3. 能够在llama-cpp-python的通用聊天模式中使用工具，而无需启动两个单独的模型
这个应用程序整合了来自[杜克瑞恩杰]（https://github.com/davorrunje）在AutoGen V0.2.3中的一些代码贡献。这些对于解决tool_llama中的函数输入格式问题起到了关键作用。在这里，我想表达我的感激之情。
2. 示例
2.1 使用自动生成模型
1、安装
pip install tool_llama
2、通过llama-cpp-python启动模型服务器
python -m llama_cpp.server \
--model ../models/mlabonne/NeuralHermes-2.5-Mistral-7B/Q4_K_M.gguf \
--n_gpu_layers -1 --n_ctx 4096 --chat_format chatml
3、你可以直接向你的infer函数中插入工具
# tools
from openai import OpenAI
from pprint import pprint
from tool_llama import Use_Tool
# types
from openai import Stream
from pydantic import BaseModel, Field
from openai.types.chat import ChatCompletion, ChatCompletionChunk
from llama_cpp.llama_types import ChatCompletionRequestMessage
from typing import List, Optional, Union
class Expert(BaseModel):
"""Expert"""
name: str = Field(description="Expert name, such as Li Rui")
description: str = Field(description="Describe the expert's skills in as much detail as possible")
...
class AdvisoryGroup(BaseModel):
"""Advisory Group"""
name: str = Field(description="Name of the advisory group")
target: str = Field(description="Advisory board objective mission")
members: List[Expert] = Field(description="members")
...
toolkit = Use_Tool()
# autogen model
@toolkit.build(tools=[AdvisoryGroup], tool_choice=AdvisoryGroup, namespace="consultative_tool")
def chat_infer(messages: List[ChatCompletionRequestMessage], stream=False,
temperature: int = 0.1) -> Union[ChatCompletion, Stream[ChatCompletionChunk]]:
api_key = "NULL"
organization = "limoncc"
base_url = "http://127.0.0.1:8000/v1"
client = OpenAI(api_key=api_key, organization=organization, base_url=base_url)
response = client.chat.completions.create(
model="mistral-7b",
messages=messages, # type: ignore
temperature=temperature,
n=1,
top_p=1.0,
presence_penalty=1.1,
stop=["</s>", '<|im_end|>'],
max_tokens=3024,
seed=1010,
stream=stream
)
return response
prompt = "Generate a team of advisors, consisting of 3 experts. To solve the AI landing application as the goal."
msg = [{"role": "user", "content": prompt}, ]
response = chat_infer(msg, stream=True)
result = toolkit.execute_function(response, tools=[AdvisoryGroup])
pprint(result[0].__dict__, sort_dicts=False)
您可以得到的结果是：
{'name': 'AI Landing Application Advisory Group',
'target': 'Solving AI Landing Application Goal',
'members': [Expert(name='Expert 1', description='An expert in AI and landing page optimization'),
Expert(name='Expert 2', description='A professional with expertise in application development and user experience design'),
Expert(name='Expert 3', description='An expert in AI-based problem solving and data analysis for landing page optimization')]}
2.1 使用由OpenAI模型生成。
# openai model
toolkit = Use_Tool()
mytools, mytool_choice = toolkit.gen_tools_api_from(tools=[AdvisoryGroup], tool_choice=AdvisoryGroup)
@toolkit.inject(namespace="consultative_tool")
def tool_infer(messages: List[ChatCompletionRequestMessage], stream=False,
temperature: int = 0.1, tools: Optional[List] = None,
tool_choice: Optional[List] = None) -> Union[ChatCompletion, Stream[ChatCompletionChunk]]:
api_key = "NULL"
organization = "limoncc"
base_url = "http://127.0.0.1:8000/v1"
client = OpenAI(api_key=api_key, organization=organization, base_url=base_url)
ok = client.chat.completions.create(
model="mistral-7b",
messages=messages, # type: ignore
temperature=temperature,
n=1,
top_p=1.0,
presence_penalty=1.1,
stop=["</s>", '<|im_end|>'],
max_tokens=3024,
seed=1010,
stream=stream
)
return ok
response = tool_infer(msg, tools=mytools, tool_choice=mytool_choice, stream=False)
result = toolkit.execute_function(response, tools=[AdvisoryGroup])
pprint(result[0], sort_dicts=False)
2.1 使用OpenAI模式，您可以使用工具进行聊天。
response = tool_infer(msg, stream=True)
for part in response:
print(part.choices[0].delta.content or "", end='') # type: ignore
你会得到这个：
To create an effective team of advisors for solving the AI landing application problem, we need to consider individuals with diverse expertise and skill sets. Here's a suggested team comprising three experts:
1. Data Scientist/Machine Learning Expert: This expert should have extensive knowledge in data analysis, machine learning algorithms, and natural language processing (NLP). They will be responsible for designing the AI models that power the landing application, ensuring they are accurate, efficient, and user-friendly. A candidate like Dr. Ziad Kobti, a renowned Data Scientist and Machine Learning Expert, would be an excellent addition to this team.
2. Full Stack Developer: This expert should have experience in both frontend and backend development, as well as knowledge of integrating AI models into web applications. They will work closely with the data scientist to ensure that the landing application is developed according to best practices and user-friendly design principles. A skilled full stack developer like Angela Yu, who has worked on various projects involving AI integration, would be a valuable addition to this team.
3. UX/UI Designer: This expert should have an eye for aesthetics and usability in digital products. They will collaborate with the data scientist and full-stack developer to create an intuitive user interface that enhances the overall experience of users interacting with the AI landing application. A talented designer like Sarah Dunn, who has worked on numerous projects involving UX/UI design for AI applications, would be a great fit for this team.
Together, these three experts will bring their unique skills and perspectives to create an effective and user-friendly AI landing application that meets the desired goals.
