Took me way too long to figure out, but if you want to use a different name for your docker-compose.yaml file (ie, gitea.yaml) and still have renovate bot look into it and work you need to do the following in your renovate.json file:

{
    "$schema": "https://docs.renovatebot.com/renovate-schema.json",
    "extends": ["config:recommended"],
    "docker-compose": {
      "fileMatch": ["^path\\/to\\/dockercomposefiles\\/.*.yaml$"],
      "enabled": true
    }
}

Of course, adjust accordingly the path/to/dockercomposefiles/ bit to be a regex match from the root of your repository, and add in whatever other renovate config you have.

In my case, this allows me to store all my portainer configs and still have them looked at by renovate without having to stick them all in an ugly folder structure.

If you're wondering what the heck I'm talking about, take a look here. It's basically an automated bot to open PRs/branches (and merge them automatically if you want after checks are passed) on your git repos when stuff updates. Great when you have a few dozen projects.