
You will notice when it's working too hard because there will be problems with the audio, such as crackling, stretching and dropouts. The objective is to find a buffer size that is low enough that your CPU can handle before it has to work too hard. Higher buffer size - higher latency but more stability with the audio signal.
Lower buffer size - lower latency but a higher chance of glitches/drop-outs. Using a lower buffer size (and therefore getting less latency) gives your CPU less time to process the audio and therefore the CPU must work much harder to process the audio in that amount of time. You can adjust the buffer size for your requirements. The time this takes is, in part, the buffer size, which you can set yourself. This delay is because the computer needs time to process the audio and route it back out of your interface. When you are recording audio with your interface, you might notice a slight delay in the audio coming back to you.
This article explains the implications of buffer sizes and latency, including how different sample rates have an effect on them.