There is no mechanism built into Weaver to constrain the single page views to a specific category or other filter. There might be a plugin that allows this.
In reality, the whole next/previous links on the single page is really of limited usefulness, and is very often simply confusing to the visitor (at least in my opinion). I think the usual “workflow” for visitors is to open the blog page that lists posts (perhaps filtered to a specific category, etc.), and use those links to see specific pages.
So an option that really doesn’t limit what people say is to set the Single Page Navigation Style to None on the Post Specifics tab.