Disclaimer: The Nintendo Switch apparently isn't compliant to USB Power Delivery (USB-PD) standards and may damage the charger or even itself. The same applies to the official charger. Therefore it is safer to only use the official Nintendo Switch charger, and only to power the Nintendo Switch itself.
The charger you're using may not provide enough power for your Switch to charge quickly. This is the case with most smartphone chargers.
The official Nintendo Switch charger provides 2.6 A * 15 V = 39 W, which is used to charge the Nintendo Switch battery while supplying the extra power needed to run games at a higher framerate/resolution on TV.
By comparison, the charger for a Sony Xperia XZ from 2016 provides only 1.5 A * 5 V = 7.5 W, which is not enough to charge the Nintendo Switch battery while playing games, but is enough to charge the battery slowly while on Standby.
Meanwhile, the charger for a Microsoft Lumia 950 XL from 2015 provides 3 A * 5 V = 15 W, which may be enough to charge the Nintendo Switch slowly while playing, depending on the game.
For a more recent comparison, the iPhone 11 Pro Max (2019) used to ship with a 2 A * 9 V = 18 W charger. I have not yet tried charging the Nintendo Switch with it, though I've tried a 96 W charger for the 16 inch MacBook Pro (2019) without any discernible difference from the official charger, though I've only ever used it once for a short amount of time. The 96 W charger supports 3 A * 15 V = 45 W and 4.7 A * 20.5 V = 96.35 W among others. I'm not an electrical engineer, so I don't know if supplying a device expecting 15 V with 20.5 V instead is dangerous or not. Luckily, @Bob has an answer to that in the comments:
It is most definitely dangerous, but then the whole point of USB-PD is that the source and sink negotiate the voltage. The source will not be supplying 20 V unless the sink explicitly requests it. So, while supplying 20 V to an arbitrary 15 V device is dangerous, using a USB-PD source that supports up to 20 V with a 15 V USB-PD sink is perfectly safe.
As mentioned before in my disclaimer, just because I managed to successfully charge my Nintendo Switch with most of the chargers mentioned above, it doesn't mean it wasn't dangerous or that my Switch didn't suffer damage. It may have suffered some damage I haven't noticed yet. Therefore I recommend using the official charger, unless you have reason to believe the Nintendo Switch has become fully USB-PD compliant.