Cotton buds can also be good for smaller gaps that your finger cannot reach, and after cleaning your earphones, make sure to dry them well.
It is recommended that you avoid water because it can damage the electronics, unless you have silicone tips that you can remove and wash with water and soap, ensuring they are dry before reinstalling.