Concrete and brickwork have very low inherent insulation properties. This comes as a surprise because the perception is that these types of walls are a substantial barrier to the outside environment. While they are a substantial barrier, this is more a measure of thermal mass; The ability to store heat (or cold). Heat will travel slowly through these types of materials but eventually will penetrate. In climates with alternating hot days and cold nights this has a moderating effect giving a good internal climate.
In climates where it is consistently hot or cold the outside temperature will eventually penetrate the wall and want to keep it that way. So, energy must be expended to either heat or cool the inside environment to combat this effect. In some climate zones the code gives some credit for thermal mass. From an energy efficiency point of view it is best to design an insulated envelope with thermal mass on the inside.