lxd - can I change containers owner (privacy?)?

I'd like to use lxd on Ubuntu 16.04, but as I see, every user who can create containers, can connect to any other containers. I'd like to prohibit this.
For example, I have two users l1 and l2 both in group lxd If user l1 creates a container:

then user l2 can connect to it:

Is there any way to prevent this and separate these containers user by user, without using tools other than lxc, which came with lxd, lxd-client?
Or is it a stupid question? (This is the 5th forum, where I ask it, but still no answer)