Recently, we included pagination as a feature to Gitsecure. This became a necessity since we figured out a way to automatically import repositories from people's GitHub accounts when they install our GitHub app.
Fortunately, our backend had this factored in, from the get-go — pagination. A typical response from the server looks like the one below.
{
"count": 6,
"next": null,
"previous": null,
"results": ["some data..."]
}Because we were working with an approach that requires you to manually import repositories, I've always overlooked implementing this feature, and went: "How many repos can someone import at a go? Who has that energy?". Yes, that's how bad the experience was before. But, the good thing now, is, we've improved and we are better.
Approaching pagination from the client (browser)
As I mentioned previously, it's been factored on the server already. What's left was for me to do this and present a way for people to use this feature if they have numerous repositories. For now, we return 25 repositories per request (or per page) — just the same way GitHub does, when you navigate to your repository tab on github.com.
One thing I'm glad I did was deciding to use SWR for our data-fetching/client request mutation processes. This decision was partly inspired by some of the guides I had read previously on Incident.io's Engineering blog — this article by Isaac Seymour, specifically.
So, for a start, I set out looking for libraries/packages on npm that I could integrate without stress. I strive with all my being to look for the most simple approach to accomplish a task. I am that lazy, and I think it's been helping me avoid unnecessary complexities in my journey.
The results I found were not satisfying to me concerning its ease of setup. Too many moving parts for something quite simple that I want to accomplish: In the first request, return the first page like so:
https://api-endpoint/repositories?page=1In the next request, when it is triggered by the click of a button, maybe a button that says "next", increment the page value. That's all.
A basic implementation
According to the SWR docs, this snippet below shows what the implementation of a typical pagination UI looks like.
const App = () => {
const [pageIndex, setPageIndex] = useState(0)
// The API URL includes the page index, which is a React state.
const { data } = useSWR(
`/api/import-repo?workspace=${workspace}&page=${pageIndex}`,
getRepositories()
)
return (
<>
<div>
{data.map((item) => {
return <div key={item.id}>{item.name}</div>
})}
<button onClick={() => setPageIndex(pageIndex - 1)}>Previous</button>
<button onClick={() => setPageIndex(pageIndex + 1)}>Next</button>
</div>
</>
)
}To make it more inclined towards what I was trying to achieve, I included a custom fetcher function that talks to the API route responsible for the repositories request. The snippet below shows something similar;
export const getRepositories = async (workspace: string, page: number) => {
const request = await fetch(
`/api/import-repo?workspace=${workspace}&page=${page}`,
{
method: 'GET',
}
)
return request.json()
}The query parameters, workspace and page lets me dynamically pass data from the client to the API routes, which then talks to the server. On and on like that for all other requests. This is possible because the API route accommodates it:
import { NextApiRequest, NextApiResponse } from 'next'
export default async function repositoriesApi(
req: NextApiRequest,
res: NextApiResponse
) {
const { workspace, page } = req.query
if (req.method === 'GET') {
doSomething(`url/${workspace}/?page=${page}`) // this would be a typical fetch syntax.
}
}with these snippets, I had a working feature. When I click on "Next", it gets the next information by incrementing the page index and vice versa for the prev button.
Improving UX by introducing indexes
The implementation above works fine. The component (buttons) does what is expected of them. But, I needed to provide a visual representation of the quantity of data a user has by spreading all of them into various pages, like the one in the image below;
Refactoring the component to accommodate this was necessary because, not only does it add a tiny improvement, by letting you know where you're currently at — like a beacon, the API response is index-based paginated.
That's why I can decide to just type the API endpoint in the browser URL tab and append ?page=${anyNumber} to it and get the corresponding data for that page if I'm authorized. The snippet below (from swr docs) shows something similar.
GET /users?page=0&limit=10
[
{ name: 'Alice', ... },
{ name: 'Bob', ... },
{ name: 'Cathy', ... },
...
]It is quite similar to the snippet I shared in the beginning. The difference here is that mine includes the following props: count, next, and prev, where count represents the total number of repositories, next and prev are like the controllers of the response gotten.
Now that I've established what it is that I want to accomplish with the indexes. In this case, I'll just refer to them as page numbers moving forward. The goal was to render the number of pages It'll take to render any set of data (repos) from the response.
The first mistake I made was to calculate for this using array.length. Yes, I was naive. "Why was it wrong to do so?" you might ask me. Well... It was wrong because I'd be considering only the first index and the value would be wrong.
Let's do a little arithmetic. Before we start, recall that the API returns 25 repos per index (page). So, when I decide to use the length of the array instead of the count property, we'll see how it backfires. Let's create a variable and call it totalItems okay?
We assign a value, results?.length, which is 25 to the variable, totalItems. Now, to get the exact amount of pages that'll accommodate the number of repositories a person has, let's create a variable and call it totalPages. To get this value, we'll divide totalItems by the amount of data we want per page. Let's create a variable for that and call it itemsPerPage
Now, we have a very tiny mental model of how we'd approach this; The snippet below summarizes the points from the two paragraphs above. You'll notice that I have wrapped the division operation in Math.ceil.
Why? I included it so that I can ensure that the total number of pages is rounded up to the nearest whole number. "Why do we need whole numbers?" Good question.
const itemsPerPage = 10
const totalItems = results?.length
const totalPages = Math.ceil(totalItems / itemsPerPage)As I mentioned previously, I used the array's length instead of the count property, by so doing, We agreed on a number of items we wanted to return per page which was 10. So, if you take a look at the snippet above, the value of totalPages becomes 2.5, without the Math.ceil(). What can we do with two and a half pages? Can we even create such?
The reason for using Math.ceil is to make sure that we always have enough pages to accommodate any remaining items that may not form a complete page. Just like our example we have 25 items and we were displaying 10 items — repositories — per page.
To render all 25 repos we would need 3 pages. Without rounding up using Math.ceil, you might end up with only 2 pages — 2.5 in this case, from the arithmetic — leaving the last 5 repositories unaccounted for.
So, if you saw that only 25 out of your repositories were displayed in your dashboard at a point in time. I was the cause. But, now, it is better because I've learned from that mistake. Now, we're returning the data as it should be. Take a look at this image again
I have close to about 205 repositories and that exact value is assigned to the count property in the response. Going ahead to use it in the arithmetic should return about eight point something... — 8.2 to be exact. Math.ceil rounds it up to 9, so the remaining repos can be added to a new page, that's why there are nine indexes in the image above.
Building the component, properly.
Now, that you and I have been able to get an integer that represents the number of pages/indexes that'll accommodate the total repositories, We have to render them in the DOM just as it is in the image. To do this, I had to rely on the use of Array.from() to give me a shallow-copied array from an "iterable-like" object.
I'll show you the "iterable-like object" soon. But, here's a basic overview of how it works. The snippet below is from mdn;
console.log(Array.from('foo'))
// Expected output: Array ["f", "o", "o"]
console.log(Array.from([1, 2, 3], (x) => x + x))
// Expected output: Array [2, 4, 6]Because my aim here is to render a couple of buttons in the 1 - 9 range. Remember, this range is due to the value of totalItems that we both calculated. To accomplish this, the Array.from method comes in.
{
Array.from({ length: totalPages }, (_, index) => <SomeJSX />)
}The snippet above creates an iterable-like object, the one we talked bout before, and the method helps us create an array from the value of totalPages. The second argument (_, index) is a mapping function that is applied to each element in the newly created array.
In my case, it takes the current element — since I don't have any value apart from the numbers, I can just append the underscore (_) — and the index, to return something. In this case, a Button component from ChakraUI, since that's what we use.
{
Array.from({ length: totalPages }, (_, index) => (
<Button
key={index}
color={
pageIndex === index + 1 ? 'var(--input-outline)' : 'var(--altGray)'
}
onClick={() => switchPage(index + 1)}
>
{index + 1}
</Button>
))
}The next question(s) that might pop into your head, after going through the snippet above could be: "Hey! where's pageIndex and switchPage coming from?". Chill, we'd take a look at them sooner than later. But, before we do, pageIndex is a state variable that tracks the current page you're on and the swicthPage function takes you to the exact page you click on.
One more thing you may have noticed in the snippet is how I returned {index + 1} in the button component. This is so the numbering is in a human-readable form. Computers start their numbering from 0, we count from one upwards. So the original iterable array [0, 1, 2, 3, 4, 5, 6, 7] becomes [1, 2, 3, 4, 5, 6, 7, 8].
Now, onto the state variable and the page switching/navigating function; In it, the initial value of pageIndex is set to 1, because that's what our API accommodates. Page numbering starts from 1. Yours could be zero, who knows
const [pageIndex, setPageIndex] = React.useState(1)
const switchPage = (newPageIndex: number) => {
if (newPageIndex >= 1 && newPageIndex <= totalPages) {
setPageIndex(newPageIndex)
}
}An important thing to also note is how switchPage uses a control flow to check if the newPageIndex is within valid bounds to ensure that the page index remains within a valid page range. So when you click on any page number, it updates the exact state.
Below is a snippet of code showing the important parts of what it looks like. I'm using useRepositories a custom SWR data hook that handles the request for me. It accepts the page index and the current workspace as arguments. To add a bit of visual cue to the process, I'm also updating the current page index in the browser URL.
export const Incidents = () => {
const router = useRouter()
const [pageIndex, setPageIndex] = React.useState(1)
const { isLoading, isValidating, repositories } = useRepositories(
workspace as string,
pageIndex
)
const { count, results } = repositories || {}
const incidentsPerPage = results?.length
const totalItems = count
const totalPages = Math.ceil(totalItems / incidentsPerPage)
const switchPage = (newPageIndex: number) => {
if (newPageIndex >= 1 && newPageIndex <= totalPages) {
setPageIndex(newPageIndex)
router.replace(
{
pathname: router.pathname,
query: { page: newPageIndex },
},
undefined,
{ shallow: true }
)
}
}
return (
<>
{totalPages > 1 ? (
<Center my="1em">
<Stack direction="row" spacing={2} mt={3} mb={5}>
{Array.from({ length: totalPages }, (_, index) => (
<Button
key={index}
color={
pageIndex === index + 1
? 'var(--input-outline)'
: 'var(--altGray)'
}
onClick={() => handlePageClick(index + 1)}
>
{index + 1}
</Button>
))}
</Stack>
</Center>
) : null}
</>
)
}Another nice hack you'll notice is how I'm also rendering the buttons, conditionally. Here's how it works: if the totalPages variable is greater than 1, render the buttons, if not, don't show anything.
Side effects
Ah Yes! the first side effect or feedback I got from the team was how, sometimes, the request could be triggered and the page number is undefined. Perhaps, because of a delayed render of the Incidents component. To fix this, we had to update the API route to fall back to page one if there's no available state yet.
const request = await fetch(`endpoint/repository/?page=${page || 1}`, {
method: 'GET',
})Data is kinda unique to workspaces in the dashboard. So let's say you have a workspace with a couple of paginated data and you've navigated to say, page 7. When you switch from that workspace to a workspace with just one page — no pagination, It'll take the page index from that workspace and try to request for such data and you know how that goes, no?
A very big 404 in the request's status code. "Enough story, how then were you able to fix this?" You might ask. Well, to fix this, I needed to know what the state depends on, for us, it is the workspace.
When I figured out that the dependency of the pageIndex state is the workspace, I needed to inform React to return the state to its default value whenever that dependency changes. Luckily for me, from the get-go, tiny UI states like these are in the URL.
So, all I needed to do was to get the workspace parameter from the URL and update the page index when there's a workspace change.
import { useRouter } from 'next/router'
export const Incidents = () => {
const router = useRouter()
const { workspace } = router.query
const [pageIndex, setPageIndex] = React.useState<number>(1)
React.useEffect(() => {
setPageIndex(1)
}, [workspace])
return <CorrespondingJSX />
}Final thoughts
As time goes on, I'll try to improve this feature so it accommodates all the edge cases I have in mind when I'm chanced. One particular enhancement would be to fully persist the state in the URL, as it is with the workspace state. So watch out for this in future releases.
If your current implementation does not involve swr, you can try this package, @ajna/pagination, it may fit your need. It didn't do the trick for me, because it had so many moving parts and I wanted to learn how this feature works under the hood.