Mumble is an open source software application typically used for audio communications, in particular for online gaming. It an interesting option because it is free & open source, easy to host your own server (called Murmur) anywhere, clients are easily available for various platforms, and it uses the low-latency (2.5ms) Opus codec.
I setup a Murmur server last night running on a Amazon EC2 virtual machine in the North California (us-west-1) region. I have two Internet connections at my home in Palo Alto: Comcast Gigabit and a point-to-point WiFi connection through Etheric. My (round-trip) ping time to the VM is about 15-20ms via Comcast and 5-10ms via Etheric.
I have two PCs sitting next to each other with wired connections to the same ethernet switch, which connects to a rack switch, which connects to my Internet gateways. One of the PCs has a Yeti USB Microphone for input and a Meridian Prime headphone amp for output. The second PC has a Corsair Void gaming USB headset.
An initial test I tried was speaking into and tapping the microphone of one PC while listening to audio on the other. When connecting via Mumble to my EC2 Murmur server, the latency wasn’t terrible, but it was high enough to be noticeable.
I also setup a Murmur server on my LAN, with ping time from the PCs at under 1ms. I tried the same test using this server, and was surprised to find that it sounded very similar. That suggested to me that the bottleneck wasn’t in my Internet link.
Next I tried installing the Voicemeeter app from VB-Audio to both PCs. I used this with the VBAN protocol to test audio being sent directly from one PC to the other (only passing through the one switch), and it didn’t sound any different to me. I used the same software to just run a simple monitor test (analog input through DAC to analog output on the same PC). On the PC with the USB headset, it sounded similar as the other tests. On the other PC with the Yeti mic, the latency dropped significantly, to the point where I could tap quickly on the mic and feel like I was hearing my taps as they happened.
All this gets me to thinking about how problematic your audio pipeline can be: the specific DACs, software, OS kernel, etc. all seem to make a big difference. This seems to further be evidenced by all the people in pro audio forums talking about the variability of latency in their recording pipelines. Having a Raspberry Pi solution with everything pre-configured for you seems like a more promising solution.