I checked your site and as far as I can see the performance problems are mostly with the initial requests. Subsequent requests seem to be fine to me.
This is most likely caused by WP-Cron, a system where WordPress executes periodic background tasks the first time the website is opened in a while. Fortunately, you can offload that with cron jobs, like this: