The trio push their sound in new directions by mixing classic rock, R&B and hip-hop but the results don’t quite hit the mark