Building Data Services for LibreChat with React Query

I’ve been working on a project that started as a clone of ChatGPT and has evolved into a feature-rich interface for communicating with OpenAI, BingAI, and SydneyAI. In truth, I had started my own LibreChat project and got pretty far with it before discovering this project from Danny Avila on GitHub, which I found through this node api for ChatGPT that I was experimenting with from waylaidwanderer. I liked the overall direction of the project and the people involved with it, so I decided to suspend operations on my own LibreChat and start contributing to Danny’s project instead.

The LibreChat project is a React application that runs on a Node.js/express/mongodb backend and it really takes ChatGPT to the next level. In my opinion, if you find ChatGPT useful, you should be using LibreChat instead. Even if you haven’t found it useful yet in your daily life, you probably will once you start using our app. It is a much more robust and feature-rich application that is actively being developed and maintained, with new features being added regularly. If you like the project, want to get involved, or have questions or comments, be sure to join the Discord server as well. Ok, enough plugging the project, let’s get into the meat of the article.

The first thing I wanted to do for the project was clean up the data fetching code, which was sort of scattered throughout the client code. I decided to use React Query to manage the data fetching and caching, as I am a big supporter of the TanStack Query library. My goal was to put a scalable solution in place to make it easier to add new features as the project evolves and provide for better maintainability and faster debugging.

In this article I will describe the solution, the reasoning behind it, and the benefits it provides for rapid development. I will also describe the process for creating new data services using the design patterns I have established for use with React Query.

Purpose

The purpose of this work was as follows:

• To organize, consolidate, and co-locate data services to make code easier to maintain and debug
• Provide data caching of queries for improved performance
• Achieve greater flexibility in how we want the UI to respond to different states (ie. success, error, loading, data)
• Simple query invalidation and refetching from within the hook instead of having to refresh the state when data changes
• TypeScript type checking of requests/responses for faster debugging on api-related issues
• Improved testability through separation of concerns and layers of abstraction
• Cleaner code: no more fetching data in useEffect!

The project was already using the SWR library from Vercel, which does provide some of the same benefits as React Query, but React Query gives you features like automatic retries, refetch on interval, caching/suspense integration, cancelation tokens, global state management, SSR support, and more. SWR, on the other hand, focuses mainly on providing hooks for remote data fetching. SWR offers some caching features, but they are not nearly as robust as React Query. Another big difference is that React Query only re-renders components when the data they depend on has been fetched or refetched, whereas with SWR, components always re-render when new data is fetched, even if the component doesn’t rely on that data. As a rule, we generally want to avoid unnecessary re-renders whenever possible. That being said, React Query is considered to be more performant than SWR.

Design

I have used the following approach on large-scale applications with some variations depending on the needs of the project, but it has always proven to do the job and scale well as an application grows. The design involves co-locating data services into a data-provider directory (or package if you’re using a mono-repo with multiple apps and packages) which contains the following:

• request.ts: contains all of the axios request methods
• api-endpoints.ts: holds api endpoints. These are separated from axios requests because many data services use the same endpoints and we don’t want to repeat ourselves.
• data-service.ts: gets the endpoints from api-endpoints.ts and makes the requests using request methods from request.ts, and returns the promise to the react-query-service hook.
• react-query-service.ts: gets data services from data-service.ts and contains all of the hooks for using the different data services with the features provided by React Query.
• types.ts: provides type definitions

Having a separate hook for each query and mutation and co-locating them together keeps things clean and organized. This way you always know where to find the hooks for a given data service. This also makes a project more approachable for new contributors who may not be as famiilar with the codebase. Note that I have linked each file to the source code in the project so you can see how it is implemented.

React Query Service Implementation

In react-query-service.ts, we import the data services from data-service.ts and export the hooks for each query and mutation. React Query has the concept of “query keys”, which are used to identify queries and mutations. The query keys are used to cache the data and to invalidate the cache when the data changes. The query keys are also used to determine when to refetch data, as we’ll see in a moment. We consolidate the query keys into an enum at the top of the file so that we can easily reference them in the hooks. This is particularly useful when creating mutations that need to do invalidations and refetches when they complete.

export enum QueryKeys {
  messages = "messsages",
  allConversations = "allConversations",
  conversation = "conversation",
  searchEnabled = "searchEnabled",
  user = "user",
  endpoints = "endpoints",
  presets = "presets",
  searchResults = "searchResults",
  tokenCount = "tokenCount",
}

React Query has two primary hooks for fetching data: useQuery and useMutation. The useQuery hook is used for fetching data, and the useMutation hook is used for updating data.

useQuery

The useQuery hook takes a query key and a function that returns a promise. The promise can be any type of data, but it is usually an object or array of objects. The promise is resolved when the data is fetched, and the data is returned to the component. The useQuery hook also takes an optional options object that can be used to configure the query.

Here is an example of a useQuery hook for fetching a conversation by id. The query key is [QueryKeys.conversation, id], which is an array of the query key and the id of the conversation. The function that returns the promise is () => dataService.getConversationById(id). The dataService is imported from data-service.ts, and the getConversationById method is a function that returns a promise. The options object is used to configure the query. In this case, we don’t want the query to refetch when the window is focused or when the app reconnects to the internet, so we set those options to false. We also don’t want the query to refetch when the component is mounted, so we set that option to false as well.

export const useGetConversationByIdQuery = (
  id: string, 
  config?: UseQueryOptions<t.TConversation> 
  ): QueryObserverResult<t.TConversation> => {
  return useQuery<t.TConversation>([QueryKeys.conversation, id], () =>
    dataService.getConversationById(id), 
    {
      refetchOnWindowFocus: false,
      refetchOnReconnect: false,
      refetchOnMount: false,
      ...config
    }
  );
}

This query is used to fetch the conversation when a search result is clicked. In the implementation, we actually disable the query by default and call the refetch method when the search result is clicked, passing it the id of the selected search result. This is because we don’t want to fetch the conversation until the user actually clicks on the search result. We’ll take a look a how this is implemented in a little bit.

The value of using TypeScript for the react query services is that it provides type checking for the data that is both sent and returned from the query. This is very useful for debugging issues related to the api. In this case, We provide the type t.TConversation as the data type for the useQuery hook. This type is defined in types.ts. We also give this as the return type of the QueryObserverResult that comes back from the API. This way, the component that uses the hook will know what type of data to expect.

useMutation

The useMutation hook is used for updating data. In the following example, we are passing the id of the conversation that we want to update as an argument to the function. The id is being used to invalidate the query for the current conversation. The useMutation hook has 4 generic type parameters: TData, TError, TVariables, and TContext, which is essentially: response data, error, request data, and context. These get passed to the UseMutationResult return type.

export const useUpdateConversationMutation = (
  id: string
): UseMutationResult<t.TUpdateConversationResponse, unknown, t.TUpdateConversationRequest, unknown> => {
  const queryClient = useQueryClient();
  return useMutation(
    (payload: t.TUpdateConversationRequest) =>
      dataService.updateConversation(payload),
    {
      onSuccess: () => {
        queryClient.invalidateQueries([QueryKeys.conversation, id]);
        queryClient.invalidateQueries([QueryKeys.allConversations]);
      },
    }
  );
};

We use the useQueryClient() function to initiate the query client for us, which is then used to invalidate the queries using the query keys we set earlier. Finally, we call the useMutation() function, which provides a TUpdateConversationRequest payload to update the conversation. In this instance, we could actually just type it as a TConversation, since that is essentially what the TUpdateConverationRequest is - something I should probably go back and change, but you get the idea. We then specify that invalidating the queries with the QueryKeys.conversation and QueryKeys.allConversations will happen on success.

In short, this function is used to update a conversation with the given ID and then invalidate the related queries to update the cache and subsequentally, the UI.

Implementing the React Query Services in the App

In order to be able to use the react query services, we need to wrap the app in a QueryClientProvider, which is done in the main or index file. This component takes a QueryClient as a prop, which is used to configure the query client. The QueryClient is imported from react-query, and we can configure it by passing an object to the constructor. To make use of the devtools, we add <ReactQueryDevtools isInitialOpen={false} /> to the App as well. This will allow us to see the queries and mutations in the devtools for troubleshooting any api-related issues.

Query functions

When we add a query to a component, the query will run automatically when the component is rendered. We’ll use useGetSearchEnabledQuery as an example.

LibreChat uses a Meilisearch server as the search engine, which must be running for search to be enabled. When the app first starts, it has to ask the api if search is enabled, so we add this to the App component.

const searchEnabledQuery = useGetSearchEnabledQuery();

  useEffect(() => {
    if (searchEnabledQuery.data) {
      setIsSearchEnabled(searchEnabledQuery.data);
    } else if(searchEnabledQuery.isError) {
      console.error("Failed to get search enabled", searchEnabledQuery.error);
    }
  }, [searchEnabledQuery.data, searchEnabledQuery.isError]);

A useEffect hook is used to watch for changes to the data and isError properties of the searchEnabledQuery. When the query is successful, the isSearchEnabled state is set to the value returned by the api. If the query fails, an error is logged to the console.

When a query requires a value, that query will automatically refetch whenever the value changes. A good example of this is the useGetConversationsQuery, contained in the Nav component. This query is responsible for populating the list of conversations in the sidebar. The query requires a page number to fetch the list of conversations for that page. When the page number changes, the query will automatically refetch the list of conversations for the new page.

// conversations displayed in sidebar
const [conversations, setConversations] = useState([]);
// current page
const [pageNumber, setPageNumber] = useState(1);
// total pages
const [pages, setPages] = useState(1);

const getConversationsQuery = useGetConversationsQuery(pageNumber);

const nextPage = async () => {
  setPageNumber(pageNumber + 1);
};

const previousPage = async () => {
  setPageNumber(pageNumber - 1);
};

useEffect(() => {
  if (getConversationsQuery.data) {
    if (isSearching) {
      return;
    }
    let { conversations } = getConversationsQuery.data;
    // make sure the user is not trying to access a page that doesn't exist
    if (pageNumber > pages) {
      setPageNumber(pages);
    } 
    else {
      //sort by most recent
      conversations = conversations.sort((a, b) => new Date(b.createdAt) - new Date(a.createdAt));
    }
    setConversations(conversations);
  }
}, [getConversationsQuery.data, isSearching, pageNumber]);

When the user clicks to the next and previous pages, setPageNumber is called, which triggers a re-render of the component. The useEffect hook watches for changes to the data property of the getConversationsQuery and updates the conversations state accordingly.

This can also be useful for things like search queries, where the query changes based on user input, as we’ll see in a moment.

Conditional queries

As I stated earlier, query functions can be disabled by default through the QueryOptions object provided on the useQuery hook. Let’s take a look at the useSearchQuery hook for getting search results:

export const useSearchQuery = (
  searchQuery: string, 
  pageNumber: string, 
  config?: UseQueryOptions<t.TSearchResults>
  ): QueryObserverResult<t.TSearchResults> => {
  return useQuery<t.TSearchResults>([QueryKeys.searchResults, pageNumber, searchQuery], () =>
    dataService.searchConversations(searchQuery, pageNumber), {
      refetchOnWindowFocus: false,
      refetchOnReconnect: false,
      refetchOnMount: false,
      ...config
    }
  );
}

In useSearchQuery, we can see an optional third argument called config, where we can pass in the enabled option to disable the query unless certain conditions are met, as seen below.

By Default, React Query refetches queries on window focus, on reconnect, and on mount. This is generally unnecessary for our purposes and just results in extra API calls (the previous version was doing a lot that). Therefore, these options are disabled by passing refetchOnWindowFocus: false, refetchOnReconnect: false, and refetchOnMount: false into the QueryOptions object of the useQuery hook.

const searchQuery = useRecoilValue(store.searchQuery);

const debouncedSearchTerm = useDebounce(searchQuery, 750);
const searchQueryFn = useSearchQuery(debouncedSearchTerm, pageNumber, { 
  enabled: !!debouncedSearchTerm && 
  debouncedSearchTerm.length > 0 &&
  isSearchEnabled && 
  isSearching,
});

Note: we’re using recoil here to get the searchQuery from the global state, but the same can be done with a useState or context api if recoil is not being used.

We can debounce the changing of the value for something like search to prevent the api from getting hammered with requests. For this I just created a simple debounce hook that takes a value and a delay as arguments, then returns the debounced value after the delay. So as the user is typing, the request to get search results is made 750ms after the user has stopped typing. The query will also be refetched when the pageNumber value changes, as we saw earlier.

To update the UI based on the results, we can use the useEffect hook to watch for changes to the data property of the searchQueryFn query. When the query is successful, the conversations state is updated with the results of the search query.

  const onSearchSuccess = (data) => {
    setConversations(data.conversations);
    setPages(data.pages);
    setIsFetching(false);
    searchPlaceholderConversation();
    setSearchResultMessages(data.messages);
  };

  useEffect(() => {
    //we use isInitialLoading here instead of isLoading because query is disabled by default
    if (searchQueryFn.isInitialLoading) {
      setIsFetching(true);
    }
    else if (searchQueryFn.data) {
      onSearchSuccess(searchQueryFn.data);
    }
    else if (searchQueryFn.isError) {
      console.error("Failed to get search results", searchQueryFn.error);
    }
  }, [searchQueryFn.data, searchQueryFn.isInitialLoading, searchQueryFn.isError])

One thing that is important to point out here, is that we use isInitialLoading instead of isLoading to check if the query is in the loading state. This is because the query is disabled by default.

The last example I’ll use goes back to the useGetConversationById hook we saw earlier. The implementation is used for clicking on a message in the conversation section of the ChatGPT UI after performing a search query. We actually don’t want this query to run unless the user has selected a conversation from the search results, so we set enabled to false by default. When the user clicks on a search result, we call refetch() on the query, and use the promise that is returned to update the UI:

  const getConversationQuery = useGetConversationByIdQuery(message.conversationId, { enabled: false });

  const clickSearchResult = async () => {
    if (!searchResult) return;
    getConversationQuery.refetch(message.conversationId).then((response) => {
      switchToConversation(response.data);
    });
  };

You can view the full code for this implementation in the Message component in the repo.

Mutations

Mutations are really easy with React Query. The useMutation hook takes a mutation function as an argument and returns an object with a mutate function that can be called to trigger the mutation. The mutate function takes an optional argument that is passed to the mutation function.

Going back to our useUpdateConversationMutation that we saw earlier, the implementation is used to edit the title of a chat conversation that gets automatically generated by ChatGPT. The mutation function takes a conversationId and title as arguments, and returns a promise that resolves to the updated conversation object.

  const updateConvoMutation = useUpdateConversationMutation(currentConversation?.conversationId);

  const onRename = e => {
    e.preventDefault();
    setRenaming(false);
    if (titleInput === title) {
      return;
    }
    updateConvoMutation.mutate({ conversationId, title: titleInput });
  };

Since the mutation will automatically invalidate the getConversationsQuery query, we don’t need to do anything else to update the UI. The useMutation hook also returns a isLoading property that can be used to show a loading indicator while the mutation is in progress if necessary. Full code for the useUpdateConversationMutation hook can be found here.

The final example we’ll use for implementing mutations will be for the preset import feature of the LibreChat project. We’ll use this example to demonstrate how you can run additional code after a mutation has completed.

The useCreatePresetMutation hook is as follows:

export const useCreatePresetMutation = (): UseMutationResult<t.TPreset[], unknown, t.TPreset, unknown> => {
  const queryClient = useQueryClient();
  return useMutation(
    (payload: t.TPreset) =>
      dataService.createPreset(payload),
    {
      onSuccess: () => {
        queryClient.invalidateQueries([QueryKeys.presets]);
      },
    }
  );
};

The mutation function takes a TPreset object as an argument and returns a promise that resolves to an array of TPreset objects. The onSuccess callback is called after the mutation has completed successfully. In this case, we are invalidating the presetsQuery query so that the UI will be updated with the new preset.

For our purposes, the invalidation will run onSuccess which will automatically cause the presets to be updated in the global store, but can also do this manually in the UI code like so:

  const createPresetMutation = useCreatePresetMutation();

  const importPreset = jsonData => {
    createPresetMutation.mutate({...jsonData}, {
      onSuccess: (data) => {
        setPresets(data);
      },
      onError: (error) => {
        console.error('Error uploading the preset:', error);
      }
    })
  };

In this example, you can see that we are running setPresets(data) onSuccess and doing some error handling by passing an onSuccess and onError functions as the second argument to the mutate function.

Full code can be seen in the NewConversationMenu component of the repo.

Conclusion

In conclusion, React Query is a great tool for managing state in React applications and can be used very effectively with scalable design patterns like the ones described here. I hope this article has given you a good idea of how to use React Query in your own projects. Be sure to check out the LibreChat project on GitHub and join the Discord server if you have any questions or comments.