Preheat the oven to 350°F. Grease and flour an 8×8 inch baking pan; set aside.
In a medium mixing bowl, mix together the ingredients for the crumb topping with a fork; set aside.
In a medium mixing bowl, mix together the ingredients for the cinnamon swirl; set aside.
In a separate medium mixing bowl, mix flour, baking powder, baking soda, salt, and chai spice.
In a large mixing bowl, beat the butter until creamy. Add the sugars and vanilla and mix until smooth.
Add in eggs one at a time, mixing in between each egg.
Add ½ the flour and mix until just combined. Mix in buttermilk and remaining flour.
Pour ½ the batter into the pan, add cinnamon swirl, and top with the last ½ of batter. Crumble the crumb topping over the batter.
Bake for 60 minutes, or until a toothpick inserted comes out with only a few crumbs clinging to it.
Allow the cake to cool before serving.