If you stick to Conventional Commits for your commit messages, you can create tags and releases with GitHub Actions for every code change you push to a repository. This is great for automated and continuous releases!
You can find plenty of frameworks and tools to provision your AWS resources. Some of them do a great job for a specific purpose, others are more generic. Nevertheless, I do prefer to use native CloudFormation templates as much as possible.
The more projects you work on, the more streamlined your tooling gets. Hopefully. Various services using different languages have different tooling requirements, of course. A sweet Makefile can be the entry to a unified tooling interface.