I've got a bunch of DSLR gear myself and I do find it too much of a bulk and hassle, was thinking of selling it all but they are really fun to use so might keep them. But for something more portable I've put in an order for a Panasonic LX7, supposed to be great fun little camera with a great lens.
I considered a few others such as mirrorless cameras but I don't need interchangeable lens as I have my DSLR and the ones on mirrorless camera are expensive and when you consider the size some add it ain't exactly small, whereas an LX7 I can have in me jacket. RX100 gets raves about its sensor but the lens on it ain't great and its expensive IMO, something else you can consider is the new Fuji X20 supposed to be a great camera but again overpriced IMO.
I'm hoping Panasonic release a successor to the LX7 with a bigger sensor, possibly with that new sensor tech they developed, coupled with the Leica 1.4 lens it would be a beast.