I think Mora is on the ball but we'd need their questions answered to know.
One possibility is that you have SELinux enabled. Check by sudo getenforce. The podman manpage explains a bit about labels and shares for mounts. Read up on :z and :Z and see if appending either to the volumes in your compose file unlocks it.
If running rootless, your host user also obviously needs be able to access it.
Just FYI and not related to your problem, you can run docker-compose with podman engine. You don't need docker engine installed for this. If podman-compose is set up properly, this is what it does for you anyway. If not, it falls back to an incomplete Python hack. Might as well cut out the middle-man.
systemctl --user enable --now podman DOCKER_HOST=unix://${XDG_RUNTIME_DIR}/podman/podman.sock docker-compose up