I guess it's both,...
You seem to be able to go out with girls, but have more difficulties in maintaining the relationship. So probably you are trying to go out with girls which don't have compatible interests/tastes/values with your own.
Maybe you should find another type of girls to date.