How about just a new fan? I have one that has run so quietly for over a year that I have to hold my finger over the air flow to feel if the fan is running or not. Runs cool with a tiny fan. Metal case, with a couple heat sinks on it. Some cases aren't big enough to allow for heat sinks to fit properly.
As an aside, I'm running WeeWx on the RP4, and the one I'm setting up now is just in the open, lying there on the bench. I put my finger on the CPU the other day and it was rather warm, approaching a hot feeling. The monitoring readout said the CPU was at 40 C, so that's pretty warm, but I've not selected a case yet.
Amazon here in the US has a iUniker case with fan and a couple heat sinks for about $11 US. I like metal, but this one is a plastic design, and since the board and CPU are not bound to the case in any way as a heat sink, I think plastic is OK, especially if plenty of holes for fan and natural convection.
You can pull the two wires the fan is powered by and watch the temperature of the unit with the Debian software which is built in and you can turn it on to watch the measurements in the taskbar, just to see.
My own opinion is to cool it with moving air, a new case or a new fan. But my opinion only. Certainly get heat sinks on it if you don't already.
Let us know what works out.