Plant neighborhood effects on herbivore damage have been observed in many systems although few studies have assessed the different component effects of the neighborhood (conspecific neighbors vs. heterospecific neighbors) on defoliation. No earlier studies have monitored how temporal scale influences neighborhood effects. We tested hypotheses on resource concentration/dilution and associational effects in the eastern spruce budworm (Choristoneura fumiferana)–forest system over a 10-yr period across different stands including stands dominated by highly vulnerable hosts (balsam fir), stands dominated by a species with low vulnerability (black spruce), and mixed composition stands (fir and spruce). We observed persistent resource concentration effects on the primary host (balsam fir) during the increasing phase of the outbreak in balsam fir-dominated and mixed stands, while both resource dilution and associational susceptibility effects were observed on the secondary host (black spruce) and the strength of associational susceptibility increased with an increase in resource dilution. We did not observe associational effects on the primary host in mixed and secondary host-dominated stands but in stands dominated by the primary host, we observed associational resistance when the resource was highly depleted. Therefore, the complexity of neighborhood effects suggests that future studies should consider the separate effects of conspecific and heterospecific neighbors, as well as changes through time in order to predict herbivore damage in different systems, and provide better preventive and reactive strategies to manage herbivore outbreaks.