Docker on macOS with an SMB File Share

For a while now I’ve been running my whole home server stack on Docker on a Mac Mini. My setup consists of the Mac Mini and a NAS. The NAS is where the large volumes of data are stored for the Docker containers to access and serve content from.

Every now and again, my containers would stop functioning and an error message would appear with something to the effect of too many open files.

Additionally, whenever files or folders would be deleted by any of the containers I would see strange hidden files on the NAS like .smbdeleteAAA* that were the size of the files being deleted. Those often would not go away.

I believe the issue came down to two things:

1. Using macOS file share as the mount
2. File permission issues

Mounting

Instead of mounting a file share at the macOS level and then bind-mounting into containers like this:

/Volume/Share/Folder:/folder

I now mount directly in Docker Compose like this:

services:
  service:
    image: <image>
    volumes:
      - type: volume
        source: nas_mount
        target: /container_folder_name
        volume:
          subpath: SubfolderInNAS

volumes:
  nas_mount:
    driver: local
    driver_opts:
      type: cifs
      device: "//<NAS-IP>/Folder"
      o: "addr=<NAS-IP>,username=user,password=password,file_mode=0777,dir_mode=0777"

The key benefit here is that Docker now uses its own implementation of SMB, which is more stable, does not produce .smbdelete* files, and is able to immediately delete things.

File Permissions

I was unable to delete some files from the share through the containers, but after performing a recursive file permission update on the NAS, the issue was resolved. I suspect that many of these files were not created or updated by the containers. Since they were migrated from a Windows host, they may have had some unusual permissions.