Grepsr Web platform¶
The Grepsr Web Platform (codebase: zephyr-web-platform) is the customer- and admin-facing
web application for the Grepsr scraping platform. It is the user interface through which clients
create projects, configure crawlers, schedule and run reports, apply data-quality and
transformation rules, and deliver datasets — and through which internal teams manage accounts,
billing, roles, and proxies.
This section is developer-focused. It is meant to onboard an engineer who has never opened the repository and get them productive quickly: how the app is structured, how data flows from the UI to the backend microservices, how authentication and authorization work, and where the app integrates with external systems. It deliberately skips line-by-line implementation detail in favour of the durable shape of the system.
Where this fits
The web platform is the Web Application Platform referenced in the High Level Architecture. It is a thin, type-safe gateway in front of the gRPC microservices — it owns presentation, session/authorization, and request orchestration, while business data lives in the backend services.
At a Glance¶
| Aspect | Choice |
|---|---|
| Framework | Next.js 14 (App Router), React 18, TypeScript 5 (strict) |
| UI | Ant Design 5 + Tailwind CSS 3 (Tailwind preflight disabled in favour of AntD reset) |
| Server state | TanStack React Query 5 |
| Client state | Zustand 4 |
| Backend transport | gRPC (nice-grpc + Protocol Buffers) from API routes; Axios/REST from the browser |
| Authentication | Auth0 (OIDC) + JWT access/refresh tokens in cookies |
| Authorization | CASL (client-side UI gating) + Cerbos (server-side policy enforcement) |
| Feature flags | Flagsmith |
| Analytics | Amplitude (incl. session replay) |
| Realtime | Socket.io (log streaming, activity feeds) |
| Deployment | Docker (Next.js standalone) on Kubernetes (AWS EKS) |
How to Read This Section¶
| Page | What you'll learn |
|---|---|
| Architecture | Layered design, the App Router layout, the provider tree, and how the folders map to responsibilities. |
| Key Modules | The major product feature areas and where each lives in the codebase. |
| Data Flow & State | The end-to-end request path (UI → hook → service → API route → gRPC) and how server vs. client state is split. |
| Authentication & Authorization | Auth0 login, JWT refresh, middleware routing, and the CASL + Cerbos two-layer permission model. |
| Integration Points | Every external system the platform talks to and the env vars that wire them up. |
| Build & Deployment | The proto build, Docker image, and CI/CD pipeline. |
| Feature Workflows | Deep dives into individual platform features (User Invite, Partial Export, Notification Subscription, Report Configuration). |
Repository Layout (top level)¶
zephyr-web-platform/
├── app/ # Next.js App Router: pages, layouts, and /api route handlers
├── components/ # React components (atoms / molecules / organisms / pages)
├── hooks/ # Custom hooks; hooks/apiHooks/ wraps services in React Query
├── services/ # API service layer (one module per backend domain)
├── apis/ # Axios instance + interceptors (token refresh, error mapping)
├── proto/ # Protocol Buffer service definitions (source of truth for types)
├── proto-types/ # Generated TypeScript clients/types (output of `yarn proto`)
├── store/ # Zustand global state slices (app, auth, pageRef, crawlerLogs)
├── providers/ # React context providers composed in the root layout
├── lib/ # CASL ability definitions, typed clients (e.g. Redis)
├── utils/ # gRPC client factory, Cerbos client, microservice registry
├── helpers/ # Domain helpers (auth, tokens, dropdowns, search, storage)
├── hoc/ # Higher-order components (e.g. admin route guard)
├── constants/ # Shared constants and test identifiers
├── types/ # Hand-written TypeScript types (one per service domain)
├── theme/ # Ant Design theme config + colour palette
└── middleware.ts # Edge middleware: session gating and auth redirects