The kanji 構 — “structure; to mind”

How to remember 構

Arranging wood (木) to build a framework. 構いません (it doesn't matter), 結構です (that's enough/no thank you) — the two weapons of polite Japanese refusal.