ShokoServer
CLAUDE.md786 starsChanged 20 months agoMITView on GitHub
Target framework: .NET 10.0. Configurations: Debug, Release, ApiLogging, Benchmarks (server + benchmarks only; tests use Debug/Release). .editorconfig with ReSharper enforcement: - Line length: 160 characters - Modifier order: private, protected, public, internal, sealed, new, override, virtual, abstract, static, extern, async, unsafe, volatile, readonly, required, file - var preferred everywhere — csharp_style_var_elsewhere, csharp_style_var_for_built_in_types, and csharp_style_var_when_type_is_apparent all set to true (enforced as warnings in src/ paths) - Braces on new lines (csharp_new_line_before_open_brace = all) - Naming: _camelCase for instance fields, _camelCase for…
The file
We split this CLAUDE.md into the rules it gives an agent
7 rules. About 0 are common, 4 are distinctive, and 3 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈4
Distinctive · estimate
This author's own choices. Worth a look.
3
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 48
When a commit qualifies for more than one type, pick the most structural one.
- Distinctive: line 132
No cookie sessions; every request is authenticated by API key
- Distinctive: line 172
Never persisted; built from persistence models in controllers/services.
- Distinctive: line 188
Always prefer a cached repository over a direct one when both exist for the same entity.
- Distinctive: line 305
Prefer the cheapest option that can actually exercise the behaviour: plain unit tests first, then the cache-backed repositories described …
- Very specific: line 413
When a file appears, jobs execute in sequence via IJobChainBuilder / RunAfterCurrent:
- Very specific: line 501
If no release provider returns a match, the provider chain completes without creating a StoredReleaseInfo record and the file is considered …
≈ “Common” is an estimate: at least 1 in 5 of the 26,980 CLAUDE.md files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
Commands
dotnet build Shoko.Server.sln
dotnet test Shoko.Tests/Shoko.Tests.csproj --filter "FullyQualifiedName~ClassName.Method"
dotnet test Shoko.IntegrationTests/Shoko.IntegrationTests.csprojTarget framework: .NET 10.0. Configurations: Debug, Release, ApiLogging, Benchmarks (server + benchmarks only; tests use Debug/Release).
Code Style
.editorconfig with ReSharper enforcement:
- Line length: 160 characters
- Modifier order:
private, protected, public, internal, sealed, new, override, virtual, abstract, static, extern, async, unsafe, volatile, readonly, required, file varpreferred everywhere —csharp_style_var_elsewhere,csharp_style_var_for_built_in_types, andcsharp_style_var_when_type_is_apparentall set totrue(enforced as warnings insrc/paths)- Braces on new lines (
csharp_new_line_before_open_brace = all) - Naming:
_camelCasefor instance fields,_camelCasefor static fields,PascalCasefor methods/classes/properties,camelCasefor locals/parameters
Commit Messages
We follow Conventional Commits, formatted as type(scope): subject (the scope is optional).
- Subject — present/imperative tense, lowercase, no trailing period (e.g.
add stream indices, notAdded stream indices.). - Body — past tense, describing what the commit did. Optional; include it when the why isn't obvious from the subject. Markdown is allowed, but without headers — use bold text and bullet lists instead of
#headings. - Code references — wrap code in backticks in both the subject and body: methods (
`FindReleaseForVideo`), types (`VideoLocal`), endpoints/paths (`/api/v3/Series`), settings, and file names. Use the bare identifier, not a prose description. - Append
[skip ci]to the subject for changes that should not trigger CI.
Types
| Type | When to use |
feat | Adds a new user-, API-, or plugin-facing capability (endpoint, DTO field, abstraction surface). |
fix | Corrects broken or incorrect behavior. |
refactor | Restructures code without changing external behavior — renames, splits, simplifies, removes. |
chore | In-tree housekeeping with no behavior change — version bumps, import sorting, comment/doc-comment fixups. |
docs | Documentation-only changes — XML docs, READMEs, API deprecation notices. |
repo | Repository infrastructure and tooling — workflows, scripts, dependencies, devcontainer, .gitignore/.dockerignore. |
misc | Small changes that don't fit cleanly elsewhere. Reach for this only when nothing else fits. |
revert | Reverts a previous commit. |
Precedence
When a commit qualifies for more than one type, pick the most structural one. Dominance order: refactor > feat > fix > chore/repo/docs/misc.
- A commit that both adds a feature and restructures or removes existing code is a
refactor, not afeat. - Introducing a new feature that requires dropping a previous feature is a
refactor(the removal dominates).
Disambiguation
chorevsrepo—choreis in-tree code housekeeping (imports, version bumps, comments);repois the build/CI/tooling/deps that live around the code.refactorvsfix— if external behavior visibly changes for the better, it'sfix; if behavior is identical, it'srefactor.chorevsmisc— prefer a precise type;miscis the last resort.dockeris a scope, not a type (e.g.repo(docker),feat(docker)).
Scopes
Scopes are optional and free-form, but reuse the established ones where they apply (non-exhaustive): abstractions, api, db, plugin, images, relocation, anidb, tmdb, search, scrobble, core, deps, workflows, scripts, docker.
Example
refactor: remove parallel mode and simplify release search pipeline
Collapsed the dual-path release search into a single sequential
pipeline. The parallel evaluator added contention without a measurable
throughput gain on real libraries.
- Removed the parallel branch and its bespoke locking
- Folded the remaining provider lookup into `FindReleaseForVideo`Architecture
Project Layout
Shoko.Abstractions— NuGet package for plugin authors. Defines the interface contract between the core and plugins (IPlugin,IShokoSeries,IShokoEpisode,IVideo,IUser, and all service/metadata/video/user interfaces). Only update this when the plugin contract itself needs to change.Shoko.Server— All implementation: API, database, repositories, services, scheduling, providers, models.Shoko.QueueProcessor— Custom job queue engine. DefinesIQueueScheduler,IQueueJob,RecurringJobRegistry,IJobChainBuilder,IVideoReleaseProviderJob<T>, persistence (QueuedJob,JobRepositoryvia EF Core), concurrency/acquisition attributes, and the orchestration stack (QueueOrchestrator,WorkerPool,WorkerPoolManager). The server references it as a local project; it is also published to NuGet for plugins.Shoko.CLI— Headless server entry point. Instantiates and managesSystemServicedirectly.Shoko.TrayService— Cross-platform tray app (Avalonia) embedding the server. Runs on Windows, Linux, and macOS.Shoko.Plugin.LegacyPlex— Bundled first-party plugin holding the legacy Plex support ("Legacy Plex Sync Support"): the Plex client (PlexHelper, handed out byPlexHelperProvider) and models, the version-neutral/plexroutes (PlexWebhook: webhook, account linking, server and library picks, sync),/api/v3/Action/PlexSyncAll,SyncPlexWatchedStatesJoband the "Plex Sync All" scheduled action. Disabling the plugin turns all of it off. Its data stays in the core: the picked server and libraries inServerSettings.Plex, each user'sPlexTokenandPlexUsersonJMMUser. To reach them it referencesShoko.Servercompile-only, the one plugin that does; do not copy that in other plugins.Shoko.Tests— Unit tests.Shoko.QueueProcessor.Tests— Tests for the job queue.Shoko.IntegrationTests— Integration tests.Shoko.TestData— Shared test data.Shoko.Benchmarks— BenchmarkDotNet benchmarks.Shoko.BuildTools/Shoko.BuildTools.Targets— build tool and MSBuild props/targets for plugin builds (assembly metadata injection, plugin manifests).
Startup Sequence
Entry points: Shoko.CLI/Program.cs (headless) or Shoko.TrayService/Program.cs (tray app). Both instantiate new SystemService() directly, which internally builds and starts the IHost.
Program.cs → SystemService constructor (NLog, PluginManager, ConfigurationService, SettingsProvider, PluginPackageManager) → SystemService.StartAsync() (keeps every startup message in ISystemService.StartupMessages, through a failed start, until about 10 seconds after the server has started; seeds the default package repositories and installs plugin upgrades before the plugin scan, then scans for plugins and builds and starts IHost / ASP.NET Core on port 8111) → SystemService.LateStart() (database schema patches and DatabaseFixes data fixups, plugin database migrations, init queue scheduler via IQueueScheduler, UDP connection handler, file watchers).
Note: LateStart() is skipped during first-run setup mode (InSetupMode == true). It runs either on normal startup or when CompleteSetup() transitions out of setup mode.
Global service container is exposed via ISystemService.StaticServices = _webHost.Services for legacy code that predates DI,
and should not be used for new code unless DI is not an option and only as a last resort.
API Pipeline
Middleware order (configured in Shoko.Server/API/APIExtensions.cs, UseAPI()):
UseForwardedHeaders— only whenWeb.TrustedProxies(set bySHOKO_TRUSTED_PROXIES) lists IP addresses or CIDR ranges (TrustedProxies): a request from a listed peer takes theX-Forwarded-Host,X-Forwarded-ProtoandX-Forwarded-Forit sent, through every listed proxy in a chain; nothing else is trusted, loopback included. First, so everything after it, the web UI's host routing among them, sees the forwarded host, scheme and client address- Sentry exception handling (if not opted out)
DeveloperExceptionPage(DEBUG /AlwaysUseDeveloperExceptions)- Swagger UI (if enabled) — configurable path via
WebSettings.SwaggerUIPrefix UseRoutingWebUIMiddleware— serves the web UI of the request's host for requests no route took (after routing, so a mapped route always wins, and never under/api,/pluginor/signalr): theServingStateof the host's override inWebUIServiceSettings.HostOverrides, else the default one, picks theIWebUIProvider, served by aStaticFileMiddlewareat its mount path, made anew for each new serving state, plus the redirects from/and from the core web UI's path when another web UI is mounted elsewhere, and of a browser's page outside the mount path into it at the same path: at once until the server has started, otherwise once the rest of the pipeline, plugin middleware included, answered it404. A request that does not accepttext/htmlkeeps its404. Always registered; the selection, the host overrides,IsEnabled(inWebUIServiceSettings) and mount paths apply liveUseCors— any method and header; any origin unlessWeb.AllowedOrigins(set bySHOKO_CORS_ORIGINS) lists some. Before the endpoints, so every answer gets its headers, and before the not-running check, so preflights work in setup and failure modeServerNotRunningMiddleware— returns 503 until the server is started, exempted via[InitFriendly]UseAuthentication— custom "ShokoServer" schemeActorContextMiddleware— makes the request's API token the current actor (IActorContext) until the request endsUseAuthorization— policies:"admin"(IsAdmin == 1),"init"(setup user only)UseEndpoints—MapControllersand the SignalR hubs:/signalr/logging,/signalr/aggregate- Plugin middleware registration
UseMvc(legacy,EnableEndpointRouting = false)
Global action filters (registered on all MVC controllers):
DatabaseBlockedFilter— returns 400 if DB is blocked, exempted via[DatabaseBlockedExempt]MetadataProviderUnavailableAttribute(exception filter) — aMetadataProviderUnavailableExceptionanswers 502 withRetry-After(the exception's wait, else what is left of the source's suspensions); aMetadataProviderNotConfiguredException(the TMDB plugin without an API key among them) answers 503
Authentication (Shoko.Server/API/Authentication/):
CustomAuthHandlerextracts API key from:apikeyheader,Bearertoken,apikeyquery param, oraccess_tokenquery param (SignalR)- Validates against
AuthTokensRepository; buildsClaimsPrincipalwith user ID, role, device name - During first-run setup,
InitUser(synthetic admin) is used — no real auth required - No cookie sessions; every request is authenticated by API key
API versioning: v0 (version-less: auth), v2 (legacy REST, including the /Stream routes, can be kill-switched), v3 (current, all new endpoints). APIv1 has been removed. Version can be resolved from query string, api-version header, or custom ShokoApiReader. ApiVersionControllerFeatureProvider excludes disabled controllers at startup via individual flags (EnableAPIv2, EnableAPIv3). The legacy Plex routes at /plex are the Shoko.Plugin.LegacyPlex plugin's, version-neutral, and are turned off by disabling the plugin. The default API version (1.0) applies to every unversioned controller, in plugins and in the core alike, so do not change it.
Serialization: MVC uses AddNewtonsoftJson() (not System.Text.Json) with: MaxDepth = 10, ApiContractResolver (Shoko.Server/API/Resolvers), NullValueHandling.Include, DefaultValueHandling.Populate. SignalR also uses AddNewtonsoftJsonProtocol(), with the same resolver.
Plugin controllers are registered via AddPluginControllers during API setup.
SignalR (Real-time Events)
Two hubs, each carrying its authorization as attributes on the hub class:
LoggingHub(/signalr/logging, admins only;[InitFriendly]and[DatabaseBlockedExempt], so in setup and failure mode anyone, as the init user) — streams buffered server logs to connecting clients, separate from the aggregate hub because it can become noisy, fast.AggregateHub(/signalr/aggregate, any authenticated user) — subscription only; clients callfeed.join_single/feed.join_manyetc. to subscribe to event categories, and cannot call into a feed. Every joining or leaving call answers with the connection's feeds afterwards, andfeed.joinedsends them once the connection has joined the ones itsfeedsquery named.feed.resend_initialsends a connection already on a feed its<feed>:connectedmessages again, so a client sharing one connection among several listeners can give a late one its starting state.
Event emitters (Shoko.Server/API/SignalR/Aggregate/) bridge internal domain events to SignalR: ActionEventEmitter, AiringEventEmitter, AnidbEventEmitter, AvdumpEventEmitter, ConfigurationEventEmitter, FileEventEmitter, FilterEventEmitter, GroupEventEmitter, ManagedFolderEventEmitter, MetadataEventEmitter, NetworkEventEmitter, PluginEventEmitter, QueueEventEmitter, RelocationEventEmitter, ReleaseEventEmitter, RestartEventEmitter, SuspensionEventEmitter, UserDataEventEmitter, UserEventEmitter. An emitter can refuse a user by overriding CanConnect, as the admin-only action, relocation and restart feeds do.
The feed contract lives in Shoko.Abstractions/Web/SignalR/ (IEventEmitter, base class EventEmitter), so plugins add feeds of their own to the aggregate hub. Every emitter declares its feed's Name, matched ignoring case; EventEmitterRegistry keeps the first of two emitters sharing a name and logs a warning. By convention a plugin names its feeds after itself.
ActorHubFilter, registered globally, sets the current actor for every hub method call, connect and disconnect, plugin hubs included.
Model Layers and Separation
Three distinct model layers; do not mix them.
1. Persistence models (Shoko.Server/Models/)
NHibernate-mapped entities. Organized by source:
Shoko.Server.Models.Shoko— core domain:AnimeSeries,AnimeGroup,AnimeEpisode, the*_Userdata,VideoLocaland its places/hashes,JMMUser,FilterPreset,CustomTag, and the image store (ShokoImage,ShokoImage_Entity)Shoko.Server.Models.AniDB— AniDB metadata cache:AniDB_Anime,AniDB_Episode,AniDB_Character,AniDB_Creator,AniDB_Tag, etc.Shoko.Server.Models.Airing— airing schedules:AiringChannel,AiringSchedule,EpisodeAiring,AiringScheduleSweepStateShoko.Server.Models.CrossReference— cross-reference tables linking providers (AniDB↔MAL, and AniDB↔any plugin source, TMDB's included, through the sharedCrossRef_AniDB_Metadata_*tables) and files/tags (CrossRef_File_Episode,CrossRef_CustomTag)Shoko.Server.Models.Release— release info for video files (StoredReleaseInfo, match attempts, release candidates and overrides)Shoko.Server.Models.Internal— internal tracking entities (AuthTokens,ScheduledUpdate,Versions)Shoko.Server.Models.Metadata— the rows behind the plugin metadata stores: series, seasons, episodes, movies, collections, people, credits, tags, studios, networks, relations, suggestions, the titles and overviews of every entry of any source (Metadata_Title,Metadata_Overview, held in memory byTextCache), orderings and last-refresh times
The Embedded sub-namespaces hold unmapped types: values stored inside a column (content ratings, release cross-references) or views built from other rows at runtime (seasons, cast and crew, studios).
NHibernate mappings live in Shoko.Server/Mappings/ as *Map.cs files. Schemas should be maintained to match, as they will be migrated to Entity Framework Code-First in a future version.
2. API response DTOs (Shoko.Server/API/v*/Models/)
Never persisted; built from persistence models in controllers/services.
v2/Models/— legacy APIv2 response shapes;v2/Models/legacy/holds the fewCL_*contract classes APIv2 still accepts or returnsv3/Models/Shoko/— modern response models (Series,Episode,Group,File,User, …) extendingBaseModelv3/Models/AniDB/andv3/Models/TMDB/— provider-specific response shapes; the TMDB ones, with theCrossRef_AniDB_TMDB_*link views inv3/Models/TMDB/CrossReferences/, are built on the shared stores throughv3/Helpers/TmdbCompatibility.csv3/Models/Common/— shared types (Images,Rating,Tag,Title, etc.)- Feature folders for the rest of v3:
Action,Airing,Auth,Configuration,Hashing,ImageManagement,Logging,Mylist,Ordering,Plugin,Release,Relocation,Streaming,TextManagement
3. Abstractions interfaces (Shoko.Abstractions/)
IShokoSeries, IShokoEpisode, IVideo, IUser, etc. — implemented by persistence models, consumed by plugins and services. Plugin code should depend only on these, never on concrete Shoko.Server types.
Repository Pattern
Two variants in Shoko.Server/Repositories/:
Cached/—BaseCachedRepository<T, S>loads all rows at startup into aPocoCache(vendored inShoko.Server/Utilities/PocoCache.cs, namespaceShoko.Server.Utilities). Reads areReaderWriterLockSlim-protected. Each repository builds typed indexes viaPopulateIndexes()(e.g.,_animeIDs = Cache.CreateIndex(a => a.AnimeID)). All writes go to DB then invalidate/update the in-memory cache. Use for hot data.Direct/—BaseDirectRepository<T, S>, no cache; hits DB on every call. Use for infrequently accessed or large data.
Always prefer a cached repository over a direct one when both exist for the same entity.
Repositories use primary constructors. Side effects around saves and deletes (cascading deletes, validation, local ID assignment, follow-up jobs) go in overrides of the base class's protected virtual hooks: OnBeginSave, OnSaveWithOpenTransaction, OnEndSave, OnBeginDelete, OnDeleteWithOpenTransaction and OnEndDelete.
Access pattern: Repositories are accessed via the RepoFactory static class (e.g., RepoFactory.AnimeSeries.GetByID(id)). RepoFactory is DI-registered but exposes static fields for convenience — this is a legacy pattern similar to ISystemService.StaticServices. This exists for compatibility where DI is unavailable, but DI should be used if possible.
Plugin storage: plugins do not get repositories in the core's database. What only a plugin has goes in a database of its own: services.AddPluginDbContext<TPlugin, TContext>(name) (abstractions, no EF Core reference) registers an EF Core context that PluginDatabaseRegistrar puts on SQLite in WAL mode at data/<plugin-id>/<name>.db3. The overload taking PluginDbContextOptions<TContext> names derived contexts with MySQL and SQL Server migrations; while the core runs on that server and the admin turned on Database.UseMainDatabaseForPlugins (off by default, advanced and experimental), PluginDbContextFactory puts the database in the core's own database through the core's connection settings, every table and constraint prefixed p<plugin hash>_<name hash>_ (PluginTableNaming, applied to the model and to each migration's operations), and a plugin without that server's migrations keeps its SQLite file. PluginDatabaseMigrator applies the migrations in SystemService.LateStart(), right after the core database's schema steps and data fixes (so after the first-run setup picked the backend, which is read from the settings only then), copying first (the SQLite file, or a SQL dump of the plugin's MySQL tables; SQL Server relies on per-migration transactions), and a failed migration is a start-up failure. Until then PluginDatabaseGate keeps every plugin context closed: the factory throws InvalidOperationException rather than wait, so Setup, Ready and hosted services' StartAsync cannot use one. Uninstalling with purgeData drops the plugin's prefixed tables on the next start. PluginPaths<TPlugin> (open generic singleton) gives a plugin its own configuration, data and cache folders, by the rules in PluginPathRules. The data every source shares goes through the stores in Shoko.Abstractions/Metadata/Storage/ (next to the Metadata*Data records they take, the link store among them; the link contracts and data stay in Shoko.Abstractions/Metadata/CrossReferences/), and a few services in Shoko.Abstractions/Metadata/Services/:
IMetadataCrossReferenceStore— links between AniDB entries and any source's entries, in the sharedCrossRef_AniDB_Metadata_*tables.IMetadataSeriesStore(series with their seasons and episodes),IMetadataMovieStoreandIMetadataCollectionStore— a plugin source's own entities, written by its provider during a refresh and read back whole throughIMetadataServicebyMetadataGuid. A plugin'sIMetadataResolver(declaring aMetadataEntityScopeof source and kind pairs, never a core or reserved source) is asked before the stores for the pairs it took; the first in plugin load order wins each pair.IMetadataImageContributor— adds images from a plugin's own source to entries of any source, core ones included, declared as aMetadataEntityScope. OneDownloadContributedImagesJob<TContributor>per contributor is queued whenever the core refreshes an entry's images; admins turn pairs off throughIMetadataImageContributorManager(kept inMetadataServiceSettings).IMetadataPeopleStore,IMetadataTagStore,IMetadataStudioStore,IMetadataRelationStore,IMetadataSuggestionStore— typed stores for data every source shares, keyed byMetadataGuid(source, entity type, source ID).IMetadataOrderingService(inServices/) — alternate orderings of any series: global ones under a plugin's own source, users' local ones underuser, plus the preferred ordering and hidden episodes.IMetadataService.GetMergedGenres/GetMergedStudios— merge an AniDB anime's genres and studios with those of the series and movies of its linked sources (MetadataMerger, the season view's merge); theMergeOptions(source order, AniDB at its place or first, genre limit and excluded genres, kept inMetadataServiceSettings.Merge) are read and set through it, andSeasonAnimeEntry.Genres/Studioscarry the result.IMetadataTextManager(inServices/) — keeps, chooses and gathers the titles and overviews of any entry (ChooseTitle,ChooseOverview, by the steps of aTitleSelectionorOverviewSelection;GetTitleSelectionandGetOverviewSelectionbuild the settings' steps): providers write their own entries' texts, plugins may add theirs to any entry, and users pick, add, disable and remove texts.
Scheduling
The scheduling system lives in Shoko.QueueProcessor — a database-backed job queue (SQLite/MySQL/SQL Server via EF Core QueueDbContext) with an O(1) in-memory deduplication index. Job definitions remain in Shoko.Server/Scheduling/Jobs/.
Entry point — IQueueScheduler (Shoko.QueueProcessor/Abstractions/IQueueScheduler.cs):
Enqueue<T>()— enqueue with dedup; no-op if already waiting or executing.EnqueueImmediate<T>()— max-priority enqueue; returns aTaskthat completes when the job finishes.RunAfterCurrent<T>()— registers a job to run immediately after the currently-executing job. Falls back toEnqueuewithprioritize: trueif called outside a worker context.CreateJobChain()— returns anIJobChainBuilderfor sequential chains.
Job chains — IJobChainBuilder (Shoko.QueueProcessor/Abstractions/IJobChainBuilder.cs):
Build with .Then<T>().Then<T>()... and submit with .Enqueue() (queue entry[0] normally) or .EnqueueAfterCurrent() (entry[0] after the current job, rest as a chain).
Scheduled actions — ScheduledActionService (Shoko.Server/Services/ScheduledActionService.cs, plugin surface IScheduledActionService; contract IScheduledAction in Shoko.Abstractions/ScheduledActions/):
Every recurring job of the core runs as a scheduled action (IScheduledAction, apart from IExecutableAction: global, no caller, admins only) with DefaultTriggers (interval, daily, weekly or monthly at a time in the server's time zone, at start-up, or when the queue is cleared), found in plugin assemblies by ScheduledActionRegistry, or handed out while the server runs by a plugin's IScheduledActionProvider (one per key, ID a UUIDv5 of "ScheduledActionProvider={type},Action={key}" under the plugin's ID; added ones count from when they were added, removed ones lose their row; IScheduledActionService.ActionsChanged and Providers); each row keeps its plugin's ID, so uninstalling a plugin with its data removes its rows, and at start-up the rows of actions gone for 30 days are removed unless their plugin is installed but not loaded; the admin lists, invokes (POST …/{id}, IScheduledActionService.InvokeAsync), cancels and re-triggers them under /api/v3/Action/Scheduled (…/{id}/Triggers, …/{id}/Cancel). A scheduled action takes no parameters; a run with options is an executable action's job. The triggers and last runs live in the ScheduledAction table, keyed by the same UUIDv5 an executable action's ID is derived with. A trigger firing queues a ScheduledActionJob (one per action, dedup, cancellable, with the job's progress); the next run counts from the last one, and a run missed while the server was down runs once at start-up. Core scheduled actions whose work is one queue job derive from QueueJobScheduledAction<TJob>, so a run queues that job directly. Scheduling starts with the server (ISystemService.Started).
The current actor across the queue: IJobActorAccessor stores the queuing caller's user ID and device on QueuedJob (ActorUserId, ActorDeviceName) and restores the token around the job's execution; chains and RunAfterCurrent inherit it. Start long-lived timers, loops and threads inside DetachedFlow.Suppress() so they never run as the caller who happened to start them; fire-and-forget work started for a user and meant to stay theirs wraps its delegate in ActorContext.Carry.
Recurring jobs — RecurringJobRegistry (Shoko.QueueProcessor/Scheduling/RecurringJobRegistry.cs):
Timer-based IHostedService on a fixed interval the admin cannot change; the core no longer uses it, plugins still may. Resolve it from DI and call Register<T>(interval) (from a plugin's Setup, for example). Registrations before StartAsync are armed on host boot; registrations after are armed immediately. Job types must first be registered via AddQueueJobsFromAssembly.
Concurrency attributes (Shoko.QueueProcessor/Concurrency/):
[LimitConcurrency(default, max?)]— pool-level slot cap.[DisallowConcurrentExecution]— at most one instance running at a time.[DisallowConcurrencyGroup("name")]— mutual exclusion across jobs sharing a group name.[LongRunning]— exempts the job from the watchdog timeout.[RetryPolicy]— per-type override of the retry backoff (max attempts, base delay, delay cap).IJobConcurrencyProvider— a limit for job types that cannot carry an attribute (the closed provider jobs); it caps the type inside its pool, the default one, and never opens a pool of its own.
Acquisition filter attributes — block dispatch until the condition is met:
[DatabaseRequired]— waits until the DB is initialized.[NetworkRequired]— waits until network connectivity is confirmed.[AniDBUdpRateLimited]— respects AniDB UDP rate limits.[AniDBHttpRateLimited]— respects AniDB HTTP rate limits.- The jobs of every metadata, release or airing schedule provider a suspended
ISuspensionProviderlists inHeldProviderTypesare held back (SuspensionAcquisitionFilter, fed byISuspensionService, attached to the pools of the jobs marked[ProviderJob]); the TMDB plugin suspends this way for a 429 or its 5XX circuit breaker, and the core'sAniDB UDPsuspension holds the AniDB release provider during a ban. The release and relocation services also refuse to use a held provider directly, and one whoseIsConfiguredisfalse(the AniDB release provider without credentials):IVideoReleaseService.GetAvailability/IVideoRelocationService.GetAvailabilitygive theProviderAvailability(disabled, not configured, suspended or available), shown on the APIv3 provider models.
Holds — IQueueScheduler.SetPoolHeld / SetJobTypeHeld (QueueHolds) hold a worker pool by name, or a job type by JobTypeNames.Key, by hand: its jobs wait, shown as blocked, while the rest of the queue runs, and running ones finish. Once the queue has found its pools, only a pool or job type it has can be held (an ArgumentException otherwise), while any name can be released. The server keeps them in Queue.HeldPools / Queue.HeldJobTypes across restarts and restores them with QueueHolds.Set, keeping names the queue no longer has, such as a plugin's that is not loaded, with a warning; GET /api/v3/Queue/Holds lists them, POST holds and DELETE releases, taking the same Pools and JobTypes in the body (a hold naming anything unknown is a 400 validation problem and holds nothing).
IJobFactory (Shoko.QueueProcessor/JobFactory.cs): DI-resolved single-shot execution via Execute<T>(). Used internally by the worker and by tests or services that need to run a job inline.
QueueStateEventHandler (Shoko.QueueProcessor/Events/) bridges job lifecycle events (added/started/completed) to QueueEventEmitter → SignalR clients.
The queue system lives in the QueueProcessor project, but the Shoko-specific code, like jobs or more advanced acquisition filters, is defined in Shoko.Server/Scheduling.
Plugin System
PluginManager scans the /plugins/ directory, loads each plugin's assemblies into a PluginLoadContext of its own (the server's assemblies shared, a dependency's own projects shared from the dependency, the packages it ships private, so plugins may ship different versions of one library; the scan reads each plugin in a throwaway context of its own first, which finds an assembly only another plugin exports, such as a dependency's contract, in that plugin's folder), finds IPlugin implementations via reflection, and registers their services via RegisterPlugins(IServiceCollection). InitPlugins() instantiates the plugins after the service container is available. StartPlugins() then runs every plugin's Setup and, when they all succeeded, every Ready. The plugin databases are migrated later, in LateStart(), right after the core database. CorePlugin is the built-in plugin that ships with the server. BundledPlugins.targets, imported by Shoko.CLI and Shoko.TrayService, builds the in-tree Shoko.Plugin.LegacyPlex and copies it to plugins/<project name>/ in their build and publish output. Three first-party plugins live in their own repositories and are bundled from their newest stable release: ShokoAnime/Shoko.Plugin.OfflineImporter, ShokoAnime/Shoko.Plugin.Tmdb (the tmdb source; it stamps its API key in its own release workflow) and ShokoAnime/Shoko.Plugin.WebAOM (the default relocation provider). ReleasedPlugins.targets, imported from BundledPlugins.targets, bundles them when ShokoBundleReleasedPlugins is true: it reads each plugin's published metadata/manifest.json, verifies the archive's SHA-256 and keeps it in .cache/plugins/<name>/, like MediaInfo.targets. Off by default, so a build from source never reaches for the network; the CI fetches each plugin in its own job and builds with ShokoReleasedPluginsOffline. The core works without them: it keeps the TMDB database and settings migrations, MetadataSource.TMDB, the APIv3 TMDB routes and models (reading the shared stores) and WebAOMRenamerMigration. An enabled plugin that fails any of these steps is never skipped quietly or disabled: PluginManager.MarkFailed records it (LocalPluginInfo.HasFailed, FailureReason), the remaining plugins go on, and SystemService.StartAsync halts the start with the web host up (StartupFailedException, as for a failed Setup), so an admin can disable, replace or uninstall it through the [InitFriendly] plugin routes and restart. An enabled plugin that cannot load at all fails the start the same way, unless an older installed version of it loads in its place: a type or assembly it refers to is missing, it targets a newer Shoko.Abstractions or another runtime, its dependency list is invalid, or a dependency it requires is missing, disabled or refused. It is never quietly treated as disabled. A plugin built for the legacy Shoko.Plugin.Abstractions or an older major version of Shoko.Abstractions fails the start too, so the first 6.x start never migrates the 5.x renamer settings without their provider; a legacy plugin (InternalPluginInfo.IsLegacy, no ID of its own) is replaced by an installed build of the same assembly for this server (FindLegacyReplacements). Before an enabled plugin's assembly loads, PluginReferenceChecker reads its references to Shoko.Abstractions, Shoko.QueueProcessor and Shoko.Server and fails it when a type or member is gone, which is how a plugin built for an older contract is named before any of its code runs; once every plugin is known, the same check runs against the assemblies a plugin it requires gives it (FailPluginsNotFittingTheirDependencies). An optional dependency outside the plugin's range, or missing what the plugin refers to, is not shared with it instead (FindUnsharedDependencies, before any load context exists): the plugin loads as if the dependency were not installed, falling back to its own copy of the dependency's contracts. It compares metadata with metadata and loads nothing. An export that cannot be created for a start-up step fails its plugin too (CreateExports), where GetExports afterwards only skips it. A failure that leaves no web server, such as a plugin's hosted service throwing as the host starts, is traced to the plugin by the stack trace and written to plugins/failed-starts.json (RememberFailureForNextStart): the next start leaves that plugin out, comes up in failure mode saying why, and forgets it, so the start after tries it again.
PluginPackageManager.DefaultRepositories lists the package repositories the server ships with, each under a stable key: core (enabled) and community (disabled, for an admin to opt into). SeedDefaultPackageRepositories adds one at start only if its key is not in Plugins.Updates.SeededRepositories yet, without syncing, so a repository an admin removed stays removed; IPluginPackageManager.RestoreDefaultPackageRepositories (POST /api/v3/Plugin/Package/Repository/RestoreDefaults) adds the missing ones back. A feed that moves keeps its key. A repository can be disabled (PackageRepositoryInfo.IsEnabled, PUT /api/v3/Plugin/Package/Repository/{repositoryID}): it is kept but never synced and its packages are not offered, and a default one can be shipped disabled for an admin to opt into. A repository's manifest is one package or a list of packages; an entry without a name is left out with a warning, and a package listed twice is read once. A sync of the repository, once its stale time (DefaultRepositoryStaleTime unless its own) has passed, fetches its manifest with ETag/If-Modified-Since; a repository that cannot be fetched keeps its last sync. A download is never less secure than the manifest naming it (IsDowngrade): a manifest served over https may not offer an archive or name an image over plain http; the images are left out and the archive refused, the release saying why in PackageReleaseInfo.Refusals, and a redirect from https to http is refused by name. A manifest served over http may name either. An archive may give its packed_size, the zip's own, and its unpacked_size, the sum of its files' contents (not their disk usage), in bytes (optional, written by shoko-build; PackageArchiveInfo.PackedSize and UnpackedSize, 0 when left out): an install without the free space for both, plus 10% of the unpacked size for block rounding, is refused before downloading unless Plugins.Updates.IsFreeSpaceCheckEnabled is off (SHOKO_PLUGIN_FREE_SPACE_CHECK_ENABLED), and a download announced or read as another packed size, or an archive unpacking to another size, is refused (InvalidArchiveSizeException, IsUnpackedSize). Archives are unpacked file by file (ExtractArchive), refusing an entry outside the plugin's folder. LocalPluginInfo.InstalledSize is a plugin's size on disk, measured when it is found (MeasureInstalledSize): its folder, a lone DLL with the files named after it (MeasureLoneDllSize: .pdb, .xml, .deps.json, .runtimeconfig.json, and its .thumbnail.* and .icon.*), or for the core plugin the application folder without its plugins and webui folders. A manifest, each release and each archive may carry an extensions object, passed through as is to the abstractions (Extensions) and APIv3 and never read by the core; the schema allows it at those three levels only.
Plugin upgrades are installed before the plugin scan when Plugins.Updates.IsUpgradeOnStartEnabled is on (opt-out, default on, set by SHOKO_PLUGIN_UPGRADE_ON_START_ENABLED when present; independent of IsAutoUpgradeEnabled, which upgrades while running and is set by SHOKO_PLUGIN_AUTO_UPGRADE_ENABLED). PluginPackageManager, built in the SystemService constructor and registered in the web host as that same instance, syncs the enabled repositories fetched longer than their stale time ago: a repository's own StaleTime when set, else RepositoryStaleTimeOnStart (5 minutes) or DefaultRepositoryStaleTime, whichever is shorter, then installs for each enabled plugin on disk the newest release above its newest installed version with an archive IsAbiAndRuntimeCompatible accepts (SelectUpgrades). It reads the plugins on disk with the plugin manager's own reader (IInstalledPluginReader, the scan's LoadInternalPluginInfo in a collectible load context, registering nothing), and uses CoreHttpClientFactory, which holds the core's "Default" client only. It skips plugins with a pinned version, a read-only plugins folder and an unreachable repository, and gives up after StartupUpgradeBudget (90 s), starting with what is installed. Nothing is remembered between starts and a version that fails is never passed over: it fails as any plugin does, and the admin decides. A newest version that cannot load yet (CanLoad false, such as one built for a newer server) is not a mismatch: the newest older version that can load waits in its place (PickVersionToLoad), so the newer one loads once the server is upgraded. Auto-upgrade while running (CheckForUpdates) makes the same choice (SelectUpgrades). Only installing by hand picks a release built for a newer server, which then waits that way. SkipBundledPluginUpgrades (off by default, set by SHOKO_PLUGIN_SKIP_BUNDLED_UPGRADES when present) has both automatic upgrades leave the plugins the server ships in its own plugins folder alone. The scan prefers the shipped copy over an installed one of the same version, and removes that installed copy at start unless it is pinned or kept (RemoveCopiesOfBundledVersions).
Installing a package brings its dependencies along. PackageInstallPlanner plans them (IPluginPackageManager.PlanInstall): an installed version in range satisfies a dependency, loaded or not, otherwise the newest compatible release in range, preferring the dependent's repository; cycles, conflicting ranges, missing or disabled dependencies and pins outside a range are Problems. InstallPackages(plan) installs all or nothing, through an install queue run by one worker of its own (not the job queue, so it works before the scheduler is up), InstallPackage included: installs run one at a time in order, as the actor who queued them. Each entry's ID is a UUIDv5 of every step's package, version and runtime, so asking for a plan already queued or running joins that entry and every caller gets the same result. QueueInstall queues without waiting (?immediate=true on the install route, 202 with the entry), WaitForInstall waits by ID (null when nothing has it), and a caller's cancellation only stops its own wait; CancelInstall (DELETE /api/v3/Plugin/Package/Install/Queue/{entryID}, no permission check in the service) takes a waiting entry off the queue or stops a running one, undoing what it installed. GetInstallQueue (GET /api/v3/Plugin/Package/Install/Queue) lists what waits and runs, with how many callers wait on each right now. The install events carry the entry ID and the queuing actor (PackageInstallationCancelled the cancelling one), and the plugin feed sends install-queued, install-started, install-failed and install-cancelled with the entry ID, never the actor. POST /api/v3/Plugin/Package/{packageID}/Install installs the plan (a plain 424 when it cannot), and GET on the same route returns it. SelectUpgrades holds back an upgrade an installed dependent's range refuses, and both upgrade paths plan the new dependencies of a release. A version installed as a dependency carries a .dependency marker (PluginManager.Dependencies.cs), next to .pinned and .keep; a plugin is installed as a dependency while every installed version is marked, and an install marks the new version only when the plugin is new or every installed version is marked (ShouldMarkAsDependency). Such a plugin gets no Plugins.EnabledPlugins entry and is enabled on demand (FindDemand, applied before the scan picks versions, in the pre-scan reader and on every state change): while an enabled plugin depends on it, required or optional. An enabled state of its own overrides that. Orphans, plugins installed as dependencies that nothing installed on purpose needs, are listed and removed through IPluginManager, and uninstalling a plugin through the API removes the ones it leaves behind unless keepOrphans is set. IPluginManager.PlanUninstall and PlanDisable (PluginChangePlanner) work out what either would do on a copy of the state, by the same orphan, demand and dependents rules: what is uninstalled or disabled, what stops loading, and which dependencies stay and why (GET /api/v3/Plugin/{pluginID}/Plan/Uninstall, …/{pluginVersion}/Plan/Uninstall, …/All/Plan/Uninstall, …/Plan/Disable). A project setting PluginGenerateClass gets its IPlugin class generated by Shoko.BuildTools.Targets (a partial <RootNamespace>.Plugin reading the stamped identity, extended by the plugin's own partial; with no other code, a meta plugin that only brings in its dependencies; the targets detect that, PluginGenerateClass and no Compile item outside the intermediate folder, and stamp PackageIsMeta, never set by hand, which shoko-build writes into the release entry as is_meta, and into the manifest's own is_meta when every release is one (false when any is not), and the core reads without loading the plugin into LocalPluginInfo.IsMeta, PackageManifestInfo.IsMeta and PackageReleaseInfo.IsMeta, null when unmarked), and PluginIcon/PluginThumbnail embed the plugin's images under stamped names (PackageIcon, PackageThumbnail) that the scan prefers over EmbeddedIconResourceName/EmbeddedThumbnailResourceName. A plugin may also reserve a namespace (PluginNamespace or the manifest's namespace, stamped as PackageNamespace, read into LocalPluginInfo.Namespace): a short, unique kebab-case name of 2 to 32 characters, never shoko, core or plugin, that other plugins and conventions may key off. A bad one fails the plugin, and two loaded plugins claiming one both fail the start, each naming the other (FailPluginsClaimingTheSameNamespace).
Plugins can also implement IPluginApplicationRegistration to register custom middleware via RegisterServices(IApplicationBuilder, IApplicationPaths) — invoked during UseAPI() after UseEndpoints and before UseMvc.
The server serves one web UI to each host. A plugin offers one by exporting an IWebUIProvider (Shoko.Abstractions/Web/): an IFileProvider with its own mount path, link templates and releases, configured through IWebUIProvider<TConfiguration>. WebUIService (IWebUIService) picks the default web UI, the ActiveProvider (the admin's pick in WebUIServiceSettings, else the first plugin web UI in load order that opts in through IWebUIProvider.IsDefaultCandidate, else the core's CoreWebUIProvider, the Shoko Web UI, always at /webui from the webui folder of the data path), and the web UI of each host override (HostOverrides, a host name with an optional port → a provider ID, validated on save by WebUIHosts; an entry with a port beats one without, names compare ignoring case, and a host without an entry, or whose web UI is gone or failed, gets the default). It keeps a ServingState for every web UI in use in a ServingTable, worked out again when the selection, the overrides or the server settings are saved or a provider's mount path changes, raising ActiveProviderChanged once per changed host (Host is null for the default); it installs the bundled release of every web UI in use when needed (ApplyStartPolicy, at start and when one comes into use), and lists and installs releases for /api/v3/WebUI. GET /api/v3/WebUI tells other applications where the web UI of the request's host is and how to link into it, and the release routes act on that web UI too; /api/v3/WebUI/Provider reads and sets the default, the host overrides and IsEnabled. Behind a reverse proxy, the host is the forwarded one only from a peer listed in Web.TrustedProxies.
Plugin controllers are registered via AddPluginControllers during API setup. Plugins add SignalR feeds with services.AddEventEmitter<TFeed>(). Every WebSocket endpoint lives under /signalr, a plugin's at /signalr/plugin/<namespace>/…, whatever dialect it speaks (SignalR, JSON-RPC, a protocol of its own), since that is the path reverse and development proxies upgrade; plain HTTP routes stay under /api/plugin/<namespace>/… (Shoko.Abstractions/Web/Services/README.md).
User configurations are kept per user through IUserConfigurationService (Shoko.Abstractions.Config): a plugin registers each by a kebab-case ID with services.AddUserConfiguration<TPlugin>(configurationID) (optionally a display name and description, and a schema from a file relative to DataPath or JSON text) or AddUserConfiguration<TPlugin, TConfig> for an IUserConfiguration type (a base configuration, so the configuration system serves its schema and UI), the core registers webui ("Web UI"), and an ID registered twice stops start-up. They live in the UserConfiguration table and are served under /api/v3/User/{Current|userID}/Configuration/{configurationID}, and /api/v3/User/Configuration lists the registrations.
A plugin says a service it talks to cannot take work by exporting an ISuspensionProvider (one per status, collected like every other provider) and reporting through ISuspensionReporter<TProvider>, an open-generic singleton like PluginPaths<TPlugin>. SuspensionService keeps one suspension per SuspensionKind, clears them when ResumesAt passes, and lifts liftable ones for an admin.
Configuration System
No appsettings.json exists in the repo. Configuration is code-based:
ServerSettings— primary settings class, persisted tosettings-server.jsonvia[StorageLocation]attributeConfigurationProvider<T>— generic provider usingINewtonsoftJsonConfigurationfor JSON serializationSettingsProvider— singleton accessor (ISettingsProvider.Instance) for runtime settings accessappsettings.jsonis configured as an optional overlay in the host builder but is not shipped- Settings migrations — versioned JSON transforms on the persisted settings file, in
Shoko.Server/Settings/SettingsMigrations.cs. Each entry in the_migrationsdictionary runs once (on upgrade past its key) viaMigrateSettings; append new migrations, never modify existing ones.
Telemetry
The server can export traces, metrics and its logs over OTLP, for an admin's own collector (Shoko.Server/Telemetry/README.md has the settings, the names and a Grafana Alloy example). Off by default: Telemetry.Endpoint is empty, and an endpoint alone exports nothing, since each signal has its own switch. Unrelated to Sentry.
TelemetryRuntime builds the providers by hand as soon as SystemService.StartAsync has the settings, not through the web host, so they cover plugin upgrades, setup and failure mode; a process exit flushes them within 3 seconds. The core's own sources and meters (Shoko.Server, Shoko.QueueProcessor and the names under them) are exported, and once PluginManager raises PluginsInitialized, those named after each active plugin's assembly. Instrument with the runtime's own ActivitySource and Meter: Shoko.Abstractions and Shoko.QueueProcessor take no OpenTelemetry dependency, and nothing costs more than a check until something listens.
- Secrets. Never put a path, a URL, a job's arguments, a command's arguments or an exception's message in a span or metric;
error.typetakes the exception's type. No outbound request gets atraceparent(NoOutputTextMapPropagator, andCoreHttpClientFactory's handler).TelemetryRedactionProcessorandOtlpLogTargetscrub what is left as a last guard, not as the plan. The one exception is the HTTP client instrumentation'surl.full: traces turn off the runtime's redaction (System.Net.Http.DisableUriRedaction) to show what was asked for, soTelemetrySecretScrubber.ScrubUrlhides the values of parameters by name, and a request never puts a secret in its query under a name that does not say so. - Telemetry never fails the work. Helpers catch what a listener throws, and end a span first thing in a
finallythat can throw. - Jobs. One root span per attempt (
shoko.job {JobType}), sampled byJobTraceSampleRatio, linked throughQueuedJob.TraceParentto the span that queued it rather than parented to it. Queuing a job inside a span reportsshoko.job.queue {JobType}under it (QueueTelemetry.ReportQueued, fromQueueScheduler.BuildContext); outside one it reports nothing, so background work does not start a trace per job. Job metrics ignore sampling. - Database.
DatabaseSpansEnabledswaps inTracingBatcherfor session factories built afterwards, adds the EF Core instrumentation, and turns on therepository.*spans ofBaseCachedRepository'sPopulate,SaveandDelete.CaptureDatabaseStatements(on by default) keeps each statement on its span, withDatabaseTelemetry.SanitizeStatementreplacing the literals in hand-written SQL by?. Never add spans to its reads, which are in-memory lookups run thousands of times per request:PocoCacheandPocoIndexcount them instead (RecordRepositoryRead), in theshoko.repository.readsmetric and as a tally on the local root span, written byRepositoryReadsProcessor. DirectCreateCommandcalls in schema steps and data fixes, and the*WithOpenTransactionmethods, are not covered. - Logs.
OtlpLogTargetbuffers fromInitLogger, then replays with the original timestamps or drops the buffer once the settings are read.
Testing
- Framework: xUnit v3 (
xunit.v3) — there is noXunit.DependencyInjection; test classes take their dependencies through fixtures or build them directly - Mocking: Moq
- Coverage: coverlet
- Test SDK: Microsoft.NET.Test.Sdk
Where a test belongs
| Project | Scope |
Shoko.Tests | Unit tests. No database, no network, no DI container. |
Shoko.QueueProcessor.Tests | The EF Core job queue, against in-memory SQLite. |
Shoko.IntegrationTests | Full server bootstrap against a real database, run in CI over SQLite, MySQL and SQL Server (selected by DB_TYPE). |
Shoko.TestData | Shared JSON fixtures consumed by tests and benchmarks. |
Prefer the cheapest option that can actually exercise the behaviour: plain unit tests first, then the cache-backed repositories described below, and a real database only when persistence itself is the subject.
Testing code that reads RepoFactory
Domain models resolve their navigation properties through the RepoFactory statics, which normally
forces a database. Shoko.Tests/Infrastructure/ avoids that:
CachedRepo.Build<TRepo, TKey, TEntity>(keySelector, entities)returns a real repository whose rows live in an in-memoryPocoCache. Read paths, including each repository's own indexes, run exactly as in production.Save/Deleteare not supported — mock those instead.RepoFactoryScopeinstalls repositories into theRepoFactorystatics and restores them on dispose. ItsWith<TRepo, TKey, TEntity>(...)overload builds and installs in one step.
Those statics are process-global, so every test using RepoFactoryScope must be annotated
[Collection(nameof(RepoFactoryCollection))], which serialises them while the rest of the suite
keeps running in parallel.
Note that ISystemService.StaticServices is write-once per process — it throws on a second
assignment. Nothing in Shoko.Tests sets it, and new tests should keep it that way.
Database Migrations
Supported backends: SQLite (default), MySQL/MariaDB, SQL Server, selected via DatabaseFactory. Each backend keeps its own schema steps in the _patchCommands list of Shoko.Server/Databases/SQLite.cs, MySQL.cs and SQLServer.cs; data fixups live in Shoko.Server/Databases/DatabaseFixes.cs. Every step is a DatabaseCommand(version, revision, …), and the applied ones are recorded in the Versions table.
- Append new steps; never modify existing ones.
- One SQL statement per step. Two drops are two revisions, never
"…; …"in one string. - One schema version per release: add revisions to the version the release already opened rather than bumping it again.
- A string column holding an identifier (
ProviderID,EntityIDand the like) is compared exactly on every backend, as the code does in memory: give itCHARACTER SET utf8mb4 COLLATE utf8mb4_binon MySQL andCOLLATE Latin1_General_100_BIN2on SQL Server in itsCREATE TABLE. SQL Server refuses to compare a column pinned this way with one that is not, so pin both sides of any join. Names, titles and other prose keep the default collation. - A schema change done in code is a
private static Tuple<bool, string?> Step(object connection)step, which runs in order with the other schema steps. Avoid(Action) step is a post-database fix: it only runs after every schema step, with the new NHibernate mappings already in use, so keep those for data fixups. - Upgrades are only supported from v5. A pre-v5 data fixup may be retired by turning its step into a bare
new(version, revision)(a no-op that keeps the slot), but a step that drops tables is never retired.
Domain Model Relationships
File → Location
VideoLocal is the canonical record for a unique file, identified by its ED2K hash + file size. It holds hashes, MediaInfo and the import date. It does not store a path.
Note: VideoLocal.MediaInfo is serialized using MessagePack via a custom NHibernate type (MessagePackConverter<MediaContainer>), not JSON.
Note: FilterPreset.Expression and FilterPreset.SortingExpression use a custom NHibernate type (FilterExpressionConverter) for JSON serialization.
VideoLocal_Place stores where a VideoLocal physically lives: a ManagedFolderID + RelativePath. One VideoLocal can have multiple places (the same file duplicated across folders). The absolute path is computed at runtime as folder.Path + place.RelativePath.
ShokoManagedFolder (formerly ImportFolder) is a root directory Shoko monitors. Each folder has IsWatched, IsDropSource, and IsDropDestination flags used by the file relocation system.
ShokoManagedFolder (1) ──< VideoLocal_Place >── (1) VideoLocal
│
VideoLocal_HashDigest (CRC32/MD5/SHA1)File → Episode
StoredReleaseInfo is the source of truth for a file's episode mapping. It caches the full response from a release provider for a given ED2K hash + file size, including:
- Precise episode percentage ranges
- Release group, video/audio codec, language, and quality information
- Cross-references to anime/episodes
A single file can map to multiple episodes (e.g., a combined OVA file). Multiple files can map to the same episode (alternative releases).
CrossRef_File_Episode is a backwards-compatible join table kept in sync with StoredReleaseInfo. It stores a simplified view of the same mapping, keyed by ED2K hash + file size (not VideoLocalID):
Percentage— what fraction of the episode this file covers (100 for a single-episode file, 50 for a 2-part release)EpisodeOrder— which part this file is in a multi-file episodeIsManuallyLinked— indicates the user was involved in creating this link
Because StoredReleaseInfo captures precise percentage ranges rather than a single percentage value, it should be treated as the authoritative mapping.
VideoLocal (hash+size) ──< CrossRef_File_Episode >── AniDB_Episode
│ │
StoredReleaseInfo ───────────┘Episode → Series → Group
AniDB_Episode is the raw AniDB cache (episode number, type, air date, synopsis, rating). It has no Shoko-specific data.
AnimeEpisode wraps one AniDB_Episode and adds Shoko state: hidden flag, title override, and the FK to AnimeSeries. All user watch data is stored in AnimeEpisode_User.
AniDB_Anime is the raw AniDB cache for a series (titles, synopsis, ratings, episode counts, external IDs for streaming services). One AnimeSeries maps to exactly one AniDB_Anime via AniDB_ID.
AnimeSeries is Shoko's local wrapper around an AniDB anime. Adds name/description overrides, language preferences, per-source auto-link flags, and missing episode counts. All user ratings live in AnimeSeries_User.
AnimeGroup is a container for series, supporting arbitrary nesting (groups within groups via AnimeGroupParentID). Groups can be auto-named from their main series or manually named. AllSeries and AllChildren are recursive traversals.
AnimeGroup (self-referential parent ──< children)
└──< AnimeSeries (1:1 AniDB_Anime)
└──< AnimeEpisode (1:1 AniDB_Episode)
└──< CrossRef_File_Episode >── VideoLocalSeries/Episode → plugin sources
AniDB entities are connected to other sources through cross-reference tables, not direct FKs. Every plugin source, TMDB and AniList included, shares one set of tables, keyed by MetadataSource and the source's own ID as a string, and written through IMetadataCrossReferenceStore (the APIv3 CrossRef_AniDB_TMDB_Show/_Movie/_Episode classes are read-only views over them):
CrossRef_AniDB_Metadata_Series—AniDB_Anime↔ a provider's seriesCrossRef_AniDB_Metadata_Movie—AniDB_Anime↔ a provider's movie, kept against the AniDB episode standing for itCrossRef_AniDB_Metadata_Episode—AniDB_Episode↔ a provider's episode
One anime can match multiple TMDB shows (e.g., split-cour series on TMDB) and one TMDB show can match multiple anime. A provider's series, seasons, episodes, movies, collections, people, studios, networks, tags and texts live in the shared Metadata_* tables, written through the stores; the TMDB plugin keeps nothing else. A plugin source may keep models of its own in its own database (AddPluginDbContext). Images from every source are stored as ShokoImage rows, linked to their entities through ShokoImage_Entity.
Import Pipeline
Job Chain
When a file appears, jobs execute in sequence via IJobChainBuilder / RunAfterCurrent:
File appears on disk
│
▼
ScanFolderJob (Shoko.Server/Scheduling/Jobs/Shoko/ScanFolderJob.cs)
Walks managed folder, drops the records of files that are gone, and queues hashing for new or changed files
│
▼
HashFileJob (Shoko.Server/Scheduling/Jobs/Shoko/HashFileJob.cs)
Computes ED2K (primary), MD5, SHA1, CRC32 via IVideoHashingService
Only now saves the VideoLocal and its VideoLocal_Place: a VideoLocal never exists without its ED2K
│
▼ [VideoReleaseService builds a chain via IJobChainBuilder]
ProcessReleaseProviderJob<TProvider> (Shoko.Server/Scheduling/Jobs/Shoko/ProcessReleaseProviderJob.cs)
One closed job type per release provider, implementing IVideoReleaseProviderJob<TProvider>
Asks the provider for the video's release; creates CrossRef_File_Episode + StoredReleaseInfo
─── or, for a provider with a dedicated job class ───
AnidbProcessFileJob (Shoko.Server/Scheduling/Jobs/Shoko/AnidbProcessFileJob.cs)
Subclass of ProcessReleaseProviderJob<AnidbReleaseProvider> adding AniDB UDP limits
Adds file to AniDB MyList (unless skipped)
│
▼
FinalizeReleaseSearchJob (Shoko.Server/Scheduling/Jobs/Shoko/FinalizeReleaseSearchJob.cs)
Always the last entry in every provider chain
Fires IVideoReleaseService.SearchCompleted; triggers relocation if configured
│ [on new AnimeID — enqueued by provider jobs]
▼
GetAniDBAnimeJob (Shoko.Server/Scheduling/Jobs/AniDB/GetAniDBAnimeJob.cs)
Fetches full AniDB_Anime + all AniDB_Episode records via AniDB HTTP API
Creates AnimeSeries + AnimeGroup if they don't exist (CreateSeriesEntry=true)
│ [unless SkipSupplementaryUpdate]
▼
SupplementaryMetadataScheduler.ScheduleForAnime (Shoko.Server/Services/SupplementaryMetadataScheduler.cs)
Asks every registered metadata provider for the anime (also called when a release links a file)
│ [per metadata provider, the TMDB plugin's among them]
▼
SearchMetadataJob<TProvider> (Shoko.Server/Scheduling/Jobs/Metadata/SearchMetadataJob.cs)
For a source the anime is not linked on: asks the auto-linker's FindAutoLinks for its scored
candidates (TMDB searches by title, dates and episode count), then MetadataLinkingService
refuses what may not be linked, logs every candidate and links the rest (a person's search
of one anime replaces its links on that source once something new is written)
│
▼
RefreshMetadataJob<TProvider> (Shoko.Server/Scheduling/Jobs/Metadata/RefreshMetadataJob.cs)
One job per linked series, movie or collection, refreshed through the provider
(writing the shared Metadata_* stores, titles and overviews included); an entry
linked from several anime is queued once, and a series refresh queues the
episode matching of every anime linked to it
│
▼
DownloadMetadataImagesJob<TProvider> (Shoko.Server/Scheduling/Jobs/Metadata/DownloadMetadataImagesJob.cs)
Links the provider's image candidates and queues DownloadImageJob for the wanted onesOrchestration Pattern
Jobs do not use a central orchestrator. Each job enqueues its successor via IQueueScheduler.RunAfterCurrent<T>() or IJobChainBuilder. The provider job chain is built by VideoReleaseService using CreateJobChain(): one entry per enabled IReleaseInfoProvider (using the provider's dedicated IVideoReleaseProviderJob<TProvider> class if registered, otherwise the closed ProcessReleaseProviderJob<TProvider> that PluginManager registers for it through ReleaseProviderJobs), with FinalizeReleaseSearchJob appended as the terminal step. Provider jobs read the AnimeID from the release info and enqueue GetAniDBAnimeJob when a new anime is encountered.
Import sweep. What the live pipeline missed is caught by scheduled actions of their own, run by hand or on the triggers an admin sets. The legacy RunImport routes queue these in the old import's order through LegacyScheduledActions.InvokeImport (Shoko.Server/API/LegacyScheduledActions.cs), logging any that fails and going on: "Hash Unhashed Files", "Scan Managed Folders" (IVideoService.ScheduleScanForManagedFolders(): drop sources in full, the rest for new files), "Search for Metadata Matches" (every source that auto-links), "Purge Expired Orphaned Metadata", "Download All Images", "Check for Previously Ignored Files" and "Check AniDB File Updates" (release searches for files without an episode, then the anime those files need). "Import New Files" scans for new files only.
ScanForMissingReleaseInfoJob (Shoko.Server/Scheduling/Jobs/Actions/ScanForMissingReleaseInfoJob.cs) runs daily as the ScanForMissingReleaseInfoAction scheduled action; it finds StoredReleaseInfo records with unknown source or missing audio/subtitle languages and re-queues the appropriate provider job on each provider's backoff schedule (GetRescanDelay()).
RefreshAnimeAiringSoonAction (Shoko.Server/Actions/AniDB/RefreshAnimeAiringSoonAction.cs, "Refresh Anime Airing Soon") runs every 24 hours by default (one hour at the least); it reads IAiringScheduleService.GetAiringsInRange for the next AiringSoonWindowHours (AiringScheduleServiceSettings, 24 by default; date-only AniDB entries only with AiringSoonIncludeDateOnly, counted when their UTC day overlaps the window), takes each airing's own AniDB anime and every anime linked to its series, and queues a forced online GetAniDBAnimeJob (IgnoreTimeCheck, never IgnoreHttpBans) for each one in the collection last updated from AniDB at least MinimumHoursToRedownloadAnimeInfo hours ago (one at the least), so the supplementary cascade and episode matching follow. A run while AniDB's HTTP API bans us is skipped.
Intermediate Cache Models
Several models exist solely to avoid redundant I/O or external API calls. Jobs check these before making outbound requests.
FileNameHash (Models/Shoko/FileNameHash.cs) — maps FileName + FileSize → ED2K hash. Written by VideoHashingService.SaveFileNameHash() and VideoRelocationService after a file is successfully hashed. Read by AnidbReleaseProvider as a last-resort local lookup when checking for creditless/variant files before going to AniDB.
VideoLocal_HashDigest (Models/Shoko/VideoLocal_HashDigest.cs) — stores all computed hash types (ED2K, CRC32, MD5, SHA1) for a VideoLocal as Type + Value rows. Written by VideoHashingService during HashFileJob. Read when displaying or cross-referencing file hashes without recomputing.
StoredReleaseInfo (Models/Release/StoredReleaseInfo.cs) — caches the full release provider response: ED2K + FileSize, provider ID, release URI, source (BluRay/Web/etc.), codec flags (IsCensored, IsCreditless, IsChaptered), version, and cross-references to anime/episodes. Written by IVideoReleaseService.FindReleaseForVideo() inside provider jobs. Provider jobs call GetCurrentReleaseForVideo() first — if a StoredReleaseInfo already exists for the hash, the lookup is skipped entirely. Queried by the API via GetByEd2kAndFileSize(), GetByReleaseURI(), GetByAnidbEpisodeID().
StoredReleaseInfo_MatchAttempt (Models/Release/StoredReleaseInfo_MatchAttempt.cs) — tracks per-provider match attempts for a file: ProviderName, ProviderID, AttemptCount, AttemptStartedAt, AttemptEndedAt, EmbeddedAttemptProviderNames. Written by provider jobs at the start of each attempt. Read by ScanForMissingReleaseInfoJob to apply per-provider backoff logic and skip providers that have already found a result.
AniDB_AnimeUpdate (Models/AniDB/AniDB_AnimeUpdate.cs) — one row per AnimeID, storing only UpdatedAt. Written by RequestGetAnime.UpdateAccessTime() after every successful AniDB HTTP response. Read by the same method, and by RefreshAnimeAiringSoonAction, to decide whether the local AniDB_Anime record is stale enough to warrant a new fetch. GetAniDBAnimeJob respects IgnoreTimeCheck to force a refresh past this gate.
AniDB_GroupStatus (Models/AniDB/AniDB_GroupStatus.cs) — caches AniDB GROUPSTATUS UDP responses: release group name, completion state, episode range, rating. Written by GetAniDBReleaseGroupStatusJob after a UDP RequestReleaseGroupStatus call. GetAniDBReleaseGroupStatusJob.ShouldSkip() bypasses the fetch entirely if the anime ended more than 50 days ago.
AniDB_NotifyQueue + AniDB_Message — AniDB_NotifyQueue (Models/AniDB/) is a staging table for raw AniDB notification IDs (type + ID) written by GetAniDBNotifyJob. AniDB_Message stores the fully fetched message body (sender, title, body, FileMoved/FileMoveHandled flags). AcknowledgeAniDBNotifyJob and ProcessFileMovedMessageJob drain these two tables in sequence. Plugins read and act on the messages through IAnidbNotifyService (AnidbNotifyService): events when one arrives or is handled, HandleMessage for a moved file, local read state, and acknowledging upstream, which the fetch only does by itself when AniDb.Notification_AcknowledgeOnFetch is on (off by default). Notices (notify type 2) are queued but never fetched yet.
ScheduledUpdate (Models/Internal/ScheduledUpdate.cs) — one row per ScheduledUpdateType, holding what a few background tasks must remember between runs: where GetUpdatedAniDBAnimeJob left off in AniDB's update feed (UpdateDetails), the minute EpisodeAiringNotificationService dispatched through, and when CheckAniDBNotificationsJob last sent NOTIFY (AniDB's 30-minute floor). When a job runs is up to its scheduled action, which keeps its own last run; the rows of the other types are only read once, by ScheduledActionService, to carry a pre-6.0 last run over to the action.
Unrecognized Files
If no release provider returns a match, the provider chain completes without creating a StoredReleaseInfo record and the file is considered unrecognized. The file can be linked to one or more episodes via the API or by plugins.
AVDump (AVDumpFilesJob) is an on-demand utility that submits a file's media info and hashes to AniDB for manual entry. It is unrelated to unrecognized file handling and only runs on explicit user/plugin request.
Concurrency Limits
| Job | Default Concurrency (Max Concurrency) | Reason |
HashFileJob | 2 | I/O bound |
MediaInfoJob | 2 | I/O bound |
AnidbProcessFileJob | 4, and group-limited (AniDB_UDP) | AniDB UDP rate limit |
GetAniDBAnimeJob | group-limited (AniDB_HTTP) | AniDB HTTP bulkhead |
AVDumpFilesJob | 1 (16) | AVDump resource limits |
RefreshMetadataJob<T>, SearchMetadataJob<T>, DownloadMetadataImagesJob<T> | The provider's MaxConcurrentJobs (TMDB: 4) | Capped per provider job type in the default pool |
DownloadContributedImagesJob<T> | The contributor's MaxConcurrentJobs (default 2) | Capped per contributor in the default pool |
ProcessReleaseProviderJob<T> | The provider's MaxConcurrentJobs (no limit by default) | Capped per release provider in the default pool |
RefreshAiringScheduleJob<T> | The provider's MaxConcurrentJobs (default 1) | Capped per airing schedule provider in the default pool |
SweepAiringScheduleProviderJob<T> | 1 (4) | One sweep chunk per provider at a time |
DownloadImageJob | 4 | Image download throughput |
ValidateAllImagesJob | 1 (1) | Sequential validation |
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

