Annual Statements
Common Wealth Retirement / October 2024
Goals
- Eliminate the maintenance burden of static files to ensure the platform scales without the need for manual template updates.
- Maintain high visual fidelity while preventing the technical debt of rebuilding interfaces across multiple environments.
- Ensure high-volume reliability so thousands of documents generate simultaneously without compromising core application stability.
- Guarantee absolute document accuracy to mitigate the risks of incomplete files and ensure data integrity before finalization.
To address the scalability challenges of a template-heavy PDF workflow, I architected and implemented a high-performance document generation service using Puppeteer and AWS Lambda. The previous approach relied on static fillable PDFs, which threatened to cause an exponential growth in templates as new account types were added. By transitioning to an HTML-to-PDF conversion model, I enabled the reuse of existing frontend components, avoiding the need to rebuild the entire application's visual assets from scratch while maintaining the high fidelity required by the design team. The final architecture utilizes a decoupled, event-driven pipeline where backend data is dispatched via AWS SQS to trigger a Lambda function that initializes a headless browser instance. Within this Puppeteer instance, a specialized frontend application renders the document dynamically based on the provided data. Once the rendering is complete, the service executes a print-to-PDF command, persists the resulting document to an Amazon S3 bucket, and notifies the backend of the successful generation through a return SQS queue. A significant technical challenge involved synchronizing the browser's rendering state with the Lambda execution to ensure optimal performance. Printing too early resulted in blank documents, while waiting too long would have made generating thousands of files prohibitively slow. To solve this, I developed a custom messaging protocol between the application running inside Puppeteer and the AWS Lambda environment. This protocol allowed the service to trigger the PDF generation at the exact moment the DOM was ready, maximizing throughput while ensuring 100% document accuracy. Beyond just timing, the protocol also handled the validation of incoming backend messages to ensure data integrity before the rendering process began. This shift from a rigid, file-based system to a sustainable, automated microservice allowed the platform to scale seamlessly as new account types were introduced. The final solution replaced a manual and potentially exponential maintenance burden with a flexible architecture that supports complex business rules and rapid product growth.
Outcomes
- The platform now handles an unlimited number of account types and document variations without any manual template management or file-heavy maintenance. This shift transformed an exponential growth problem into a stable, linear process that scales effortlessly with product expansion.
- Design consistency is fully maintained across all generated documents because the system shares the same source code as the main application. This eliminated the need for redundant development cycles and ensured that brand standards remain unified without any extra technical debt.
- The infrastructure reliably processes thousands of document requests concurrently without impacting the performance of the core user experience. By offloading these intensive tasks to an isolated environment, the system maintains high availability and stability even during peak traffic periods.
- Every document generated is verified for accuracy and completeness before finalization, removing the risk of blank or corrupted files. The implementation of strict synchronization and validation protocols ensures that data integrity is preserved across the entire automated pipeline.