Bartholin Cysts

Earlier I was itching a little down there and found a bumpy like thing inside my labia. I started messing with it and found another one also. Could it be a bartholin cysts? if so how do I make them go away. They do not hurt I know they are there so its driving me crazy.