is there is perfect way to get AIMessageChunk Content?

[annilq]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

const appendMessage = async (message: CreateMessage) => {
    const updateMessage = await createMessage(chat.id, message) as Message
    const messageId = uuidv4()
    await fetchEventSource("/api/chat", {
      method: "POST",
      headers: {
        "Content-Type": "application/json",
        Accept: "text/event-stream",
      },
      body: JSON.stringify({
        model: chat.model,
        chatId: chat.id,
        messages,
        messageId: updateMessage.id
      }),
      onmessage: (message) => {
        if (message.event !== "data") return;

        const eventSourceMessage = JSON.parse(message.data);

        if (!eventSourceMessage.data.chunk) return;

        if (eventSourceMessage.event === "on_chain_stream") {
          console.log(eventSourceMessage.data.chunk);

          setMessages((prevState) => [
            ...(prevState.at(-1)?.role === "user" ? prevState : prevState.slice(0, -1)),
            {
              id: messageId,
              role: "assistant",
              content: eventSourceMessage.data.chunk,
            },
          ]);
        }
      },
      onerror: (err) => {
        console.log(err);
      },
    });
  };

Description

I try to get the streamevent content from data chunk,but the content nest so deep ,i notice the the type of the chunk is AIMessageChunk so I want to init the AIMessageChunk the chunk and concat the follow chunk,finnaly I want to get content of the AIMessageChunk ,but the error is below
when onmessage called below error
TypeError: firstContent is not iterable
at mergeContent (webpack-internal:///(app-pages-browser)/./node_modules/@langchain/core/dist/messages/base.js:36:20)
at AIMessageChunk.concat (webpack-internal:///(app-pages-browser)/./node_modules/@langchain/core/dist/messages/ai.js:253:76)
at onmessage (webpack-internal:///(app-pages-browser)/./app/books/[id]/page.client.tsx:84:35)
at onLine (webpack-internal:///(app-pages-browser)/./node_modules/@microsoft/fetch-event-source/lib/esm/parse.js:73:67)
at onChunk (webpack-internal:///(app-pages-browser)/./node_modules/@microsoft/fetch-event-source/lib/esm/parse.js:55:13)
at getBytes (webpack-internal:///(app-pages-browser)/./node_modules/@microsoft/fetch-event-source/lib/esm/parse.js:11:9)
at async create (webpack-internal:///(app-pages-browser)/./node_modules/@microsoft/fetch-event-source/lib/esm/fetch.js:58:17)
try to fix it

System Info

[email protected] | MIT | deps: 12 | versions: 314
mac
node:v23.7
npm :10.9.2

[dosubot[bot]]

Hello, @annilq! I'm here to help you with any bugs, questions, or contributions you have. Let's tackle this together!

To correctly retrieve and concatenate the content of an AIMessageChunk, you should use the concat method provided by the AIMessageChunk class. This method is designed to handle the merging of two AIMessageChunk instances, including their content and other fields. Here's how you can do it:

1. Ensure that you are correctly initializing and using AIMessageChunk instances.
2. Use the concat method to merge chunks as they arrive.

Here's an example of how you can implement this in your onmessage event handler:

import { AIMessageChunk } from 'path-to-your-ai-message-chunk-module';

let finalMessage: AIMessageChunk | undefined;

const appendMessage = async (message: CreateMessage) => {
  const updateMessage = await createMessage(chat.id, message) as Message;
  const messageId = uuidv4();
  await fetchEventSource("/api/chat", {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
      Accept: "text/event-stream",
    },
    body: JSON.stringify({
      model: chat.model,
      chatId: chat.id,
      messages,
      messageId: updateMessage.id
    }),
    onmessage: (message) => {
      if (message.event !== "data") return;

      const eventSourceMessage = JSON.parse(message.data);

      if (!eventSourceMessage.data.chunk) return;

      if (eventSourceMessage.event === "on_chain_stream") {
        const chunk = new AIMessageChunk(eventSourceMessage.data.chunk);

        finalMessage = !finalMessage ? chunk : finalMessage.concat(chunk);

        if (finalMessage) {
          console.log(finalMessage.content);
        }

        setMessages((prevState) => [
          ...(prevState.at(-1)?.role === "user" ? prevState : prevState.slice(0, -1)),
          {
            id: messageId,
            role: "assistant",
            content: finalMessage.content,
          },
        ]);
      }
    },
    onerror: (err) => {
      console.log(err);
    },
  });
};

Make sure that both firstChunk and secondChunk are valid AIMessageChunk instances when using the concat method. The error "TypeError: firstContent is not iterable" might occur if you attempt to iterate over a non-iterable object, so ensure that the concat method is used as shown above ^[1][2].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other}

[annilq]

still throw the same error ,is something wrong with the server side?

below is the server code

import { NextRequest } from "next/server";
import LLMProvider from "@/utils/llms_provider";
import { Message } from "ai";
import { HumanMessage, AIMessage, SystemMessage } from "@langchain/core/messages";

export async function POST(req: NextRequest) {
 const body = await req.json();
 const { model, messages } = body;

 const [provider, modelName] = model.split("/")!;
 const llm = LLMProvider.getModel(provider, {
   model: modelName,
   temperature: 0,
   maxRetries: 2
 });

 const eventStream = llm.streamEvents(
   messages.map((message: Message) => {
     let messageData;
     switch (message.role) {
       case "user":
         messageData = new HumanMessage(message.content);
         break;
       case "assistant":
         messageData = new AIMessage(message.content);
         break;
       case "system":
         messageData = new SystemMessage(message.content);
         break;
       default:
         break;
     }
     return messageData;
   }),
   {
     version: "v2",
     encoding: "text/event-stream",
   }
 );

 return new Response(eventStream, {
   headers: {
     "Content-Type": "text/event-stream",
     "Cache-Control": "no-cache",
     "Connection": "keep-alive",
   },
 });
}

[annilq]

this error appear because the chunk content is empty,I don't understand why lots of chunk with empty content will return to frontend with langchainjs on_chat_model_stream event

[annilq]

this will get the right content,but I wonder why so much empty content return to frontend,and the response speed is so slow

if (eventSourceMessage.event === "on_chat_model_stream") {
         if (aiMessage) {
            aiMessage = aiMessage.concat(new AIMessageChunk(eventSourceMessage.data.chunk.kwargs));
         } else {
            aiMessage = new AIMessageChunk(eventSourceMessage.data.chunk.kwargs);
         }
        console.log(aiMessage?.content);
      }