You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Meta-issue. Not work — the order the other 31 work packages should be resolved in, derived
from the Depends on: graph rather than asserted.
The headline: do not work milestone-by-milestone. The milestone numbers M1–M12 come
from the paper's own table (paper.tex:1213), which is a narrative order — the sequence
the plan is best read in. The build order is different, and in several places sharply so.
Status
Everything is closed: all 31 work packages, all four risk trackers, and the correction.
This meta-issue is the last one open, and it is the record rather than the work.
COSA solves eq. (8) as a conic active-set method, warm starts along a frontier, has a public
interface, and the paper carries its results, its assumptions statement and the
characterization §20 asks for — with its one negative result stated rather than smoothed
over.
(Four issues — #29, #34, #35, #36 — stayed open for a while after their work merged, for a
duller reason worth knowing: GitHub parses Closes #A, #B as closing only #A. The keyword
has to repeat.)
One correction is open: #59, which closes #36.
It is the most important finding in the project and it arrived after the last wave merged — the failure
study had been certifying its own answers, and on one family the certificate is wrong. See
result 17, which is the one to read if you read only one.
#56 is closed as merged without ever reaching main — it merged into the wave-9 branch
after main had already taken that branch via #55 — so its commit rode into #57 instead,
which is why #57 closed three issues. Nothing was lost.
The release tag is not tagged, and #59 is now a reason as well as a preference. Everything
the tag would certify is checked except the claim #59 corrects, and tagging a release whose
paper contains a false result would be the one irreversible mistake available here.
Both milestone moments have passed.#20 solved eq. (8) through §9 Phase III's four
ingredients; #23 replaced the tangent-only treatment with §9 Phase IV's conic working-set
logic. §16.2's five analytical checks — primal solution, active set, multipliers, SOC
activity, KKT residuals — pass on #9's hand-solved instance. Across the structured families
(n = 8, seed = 0), with the iteration count before and after #23:
Sixteen results. They are the reason to read this issue rather than only the milestone
table. Results 1, 2, 10, 12 and 14 reached the paper; result 15 is the one that changed the
project's own published conclusion.
1. Eq. (6) as printed is wrong — flagged for #38.paper.tex:457 gives the leading
coefficient as ||q||^2; expanding (t + alpha*tau)^2 leaves ||q||^2 - tau^2. The two
agree only when tau = 0, which is the one case eq. (7) never takes. On a direction running
along the cone's boundary to the apex the printed formula admits a step of zero where
the true answer is t / (-tau) — a solver using it would refuse to travel along the
boundary at all. Verified against seven hand-solved intersections and 10 000 random cases.
The correction also turns the root selection from one case into three, since the leading
coefficient can vanish or go negative.
2. An iterate on the cone's boundary cannot move — this is Risk 1, and it is real. (Answered by #23; see result 10.) Two facts, both proved and both about eq. (7):
at a boundary point, a direction satisfying eq. (3) has an exact conic step of zero.
Tangency kills eq. (6)'s middle coefficient, feasibility kills its constant term, and the
leading one is ||q||^2 - (u'q)^2, which Cauchy–Schwarz puts at or above zero;
the projected steepest-descent direction always pushes tdown, by exactly lam / rho, because t appears in no linear row.
So a linear direction plus an exact conic step cannot move off the boundary, either way.
That is the curvature of the cone, not a defect in the ratio test, and it is §3.3's warning
about becoming "an SQP method with an SOC constraint" and #39 arriving together. #20 answered
with the retraction §3.3 sanctions for the prototype — tangent step, then restore the cone by
raising the free head. #23's answer is result 10: the retraction is unavoidable for any method
holding its iterates exactly on a curved boundary, but the tangent plane was the wrong thing
to step along, and giving the direction the boundary's curvature is what makes the retraction
cheap instead of the bottleneck.
3. An active cone's multiplier has exactly one degree of freedom. At a nonzero boundary
point, conic complementarity forces w = w_0 * (1, -u) — the tangent covector is the dual
direction. So w in Q collapses to the scalar nu_cone <= 0, and an active cone costs the
dual side no more than an inequality does. #23's deactivation test is a statement about that
single number — and result 11 is what that number turns out to be.
4. At the apex nothing collapses, and eq. (7) can never leave it. The tangent cone of Q becomes Q itself and the normal cone becomes -Q, so the dual test is a cone membership test — §8.1's "different tangent and normal geometry", precisely. Worse: on
eq. (7) an unjustified apex can never be released, because dropping the factor forces d_t = -lam/rho < 0 and the released direction is infeasible by arithmetic. blocked-at-apex
is a real solver status that fires on real draws.
5. rho in H = rho*I is not a tuning parameter. The multipliers are exactly rho-invariant and the direction scales as 1/rho. Do not confuse it with §8.3's degeneracy
regularization (#25), which perturbs the problem and whose cost is visible: at a stationary
point the true direction is exactly zero while the regularized one is O(delta).
6. The KKT system does not inherit the covariance's conditioning.cond(Sigma) = 1e10
and the assembled KKT matrix sits at about 10, because the tangent representation puts L
into the system as a single row and one row has no spectrum. A genuine difference from
interior-point methods, and the kind of result §17 (paper.tex:1345) asks for. It also means #28's scaling earns its keep on unit mismatch rather than on ill-conditioned covariances — 2e14 → 10 there.
7. The KKT matrix is not sparse, and cannot be made so. However sparse A is, the cone
contributes the tangent row g_0 - u'L, and u is a normalized covariance direction — dense
in every asset. One dense row and its transpose put a dense cross in the matrix. So the
sparsity a sparse LDL^T would exploit is destroyed by the structure that makes the problem
conic. #27 should read this before optimizing. The comparison of #26 found range-space
3.25× the reference for a structural reason instead: it solves an m-by-m system where
the others solve (n + m)-by-(n + m).
8. large's optimum is at the apex, and nobody designed it that way. A rank-k
covariance over n >> k assets leaves L with an (n-k)-dimensional null space, so the
minimum-risk portfolio has risk exactly zero and lam > 0 takes it. The family is an apex
instance for free — and pinning the apex block with most bounds active leaves W square and
the KKT system conditioned around 1e18.
10. H = rho*I was leaving a derivative on the table, and it was the expensive one.
§4.2 writes the direction subproblem's matrix as rho*I, and the SOCP objective is linear,
so it is tempting to conclude the Lagrangian has no curvature. It does — the cone is
curved. Writing the active factor as g(z) = ||s_1|| - s_0 <= 0, the Lagrangian's Hessian is rho*I + sum_j mu_j * G_j' * grad^2 g_j * G_j, with grad^2 g = (I - u u')/||s_1|| in the
tail block. Two things make it free to use: it is positive semidefinite (because g is
convex) so the subproblem stays uniquely solvable, and mu_j is just the head of w, a
component of a multiplier #13 already computes.
That is what "primal-dual" buys, and the phrase is not decoration — the dual variable
determines the primal step. Measured:
The multipliers used are the previous iteration's, so the scheme is a fixed point rather than
an implicit system, and the first iteration uses zero — a #23 solve and a Wave 6 solve begin
bit-for-bit identically, which is what makes every difference between them attributable. #38 should carry this: the paper writes H = rho*I and, on the evidence, should not.
11. On eq. (7), §7.4's deactivation rule provably never fires.t appears in exactly one
linear row — the cone's head — and in the objective with coefficient lam. Reading the
direction subproblem's stationarity in the t slot gives w_0 = -nu_cone = lam + rho * d_t,
which at a stationary point is lam. Result 3 then forces ||w_tail|| = |w_0| exactly. So w sits precisely on the boundary of Q with a strictly positive head: neither outside Q
nor zero. An active cone at a mean–standard-deviation optimum always contributes a genuine
active normal.
This cuts two ways. It says the rule is correctly inert here rather than absent — away from a
stationary point w_0 is free to go negative, and #23 exercises the release on a constructed
SOCP that eq. (7) cannot produce. And it removes the cycling hazard a geometric
deactivation rule would have had on this problem: there is nothing for the rule to release,
so there is nothing for §7.3 to re-acquire. #29 should know this before designing
hysteresis — the oscillation §8.2 warns about cannot be a cone activate/deactivate cycle on
eq. (7), so whatever #40's instance shows must be something else.
12. Factorization reuse works on its own metric and loses on wall clock, for a structural
reason. The share of KKT solves needing a fresh factorization falls from 98.9% to 1.4%
across the families and 98.7% to 1.5% across the randomized sweep; a whole solve typically
factorizes twice. Wall clock goes the other way: a column update against a full (n,n)
orthogonal factor costs O(n^2) while a fresh QR of an (n,m) matrix costs O(n m^2), so
updating wins only once m^2 exceeds n. The classical argument for factorization
updating quietly assumes a working set comparable in size to the problem, and an
active-set method on a portfolio runs with m well below n. Measured: 0.98× at n = 300,
1.12× at n = 500.
13. The cycle that existed was not the one anyone was watching for. §17.2 names four
remedies and §8.2 names hysteresis; none of them was the problem. The revisit counter had
been counting iterations held on a working set rather than returns to it, which reported
nine hundred revisits for a solve that was merely slow and hid a genuine two-state cycle
underneath. Corrected, the worst randomized instance alternated APEX and INACTIVE486
times: #24's branch releases an apex it cannot justify, the released direction cannot be
travelled, the step moves nothing, and §7.3 reads the same unchanged geometry and puts the
factor straight back. The fix is one line of principle — do not re-derive a status from
geometry that has not changed — and the worst return count across 200 instances went to 2.
14. Warm starting is a conditional, and reporting it as a number would have hidden it.
box(8)
box(10)
box(12)
points where the carried working set was right
+55%
+18%
+18%
points that had to correct it
−27%
−17%
−32%
overall
+44%
+9%
−7%
The per-group figures are nearly identical across instances; what differs is the mix.
Correcting a belief costs more than acquiring one — a cold solve discovers the active set on
the way in, a warm one must undo a wrong answer first and then discover it anyway. So the
condition is not "the problems are close" but "the combinatorial state is the same",
and those are different conditions: a frontier can move steadily in lam while its active
set jumps.
15. The failure-mode study's central finding was wrong for two waves, with an ablation
apparently confirming it.Superseded by result 17 — the conclusion below ("solves to
1e-11 unaided") is itself wrong, and for the same underlying reason. Kept because the
sequence is the finding.
badly scaled stalled at 5e-2. Ruiz equilibration converted the stall into an optimum at 9.7e-7. That is a clean before-and-after, it pointed at conditioning, and equilibration
became the family's mitigation and briefly the public interface's default. Then #37 exposed
the real cause: raise_free_heads required a cone head row to select its variable with a
coefficient of exactly one —
— which nothing needs, since the head is coefficient * t + h and solving head >= ||tail||
is one division. No rescaled instance satisfies it, so _heads_are_free reported False, the retraction was silently unavailable, and an iterate on the cone's boundary could not
move. That is result 2 arriving with its remedy switched off, not a conditioning failure.
With the restriction removed the family runs to completion with residuals under 1e-11 —
which the study then reported as solved. It is not solved; see result 17. What does survive
from here: §13.3's scaling rescues no family and costs 30–80% more iterations on every one,
and equilibration was perturbing the head row's coefficient rather than the conditioning.
A "diagnosed" stop is a hypothesis, not a conclusion. The ablation was sound; what it
measured was a coincidence.
16. COSA is 3–30× slower than CVXPY/Clarabel in wall clock, on every mode measured.
CVXPY spends most of its time building a problem rather than solving one, so the comparison
ought to flatter COSA and does not. Reported rather than buried — §20 asks for a
characterization of when conic active-set methods work well, which is not a claim that
they always do. Where the numbers are good is iteration and factorization counts on
sequences.
17. A relative KKT residual certified an answer that is 3.4% wrong, and this is the most
important result in the list.
badly scaled terminates reporting optimal with all five of §6's residuals between 1e-11
and 1e-15, at an objective of −0.00524. Clarabel returns −0.03956, and that point is
feasible for COSA's own feasibility check to 1e-11, with a strictly better objective. The
direction between them is a feasible descent direction with derivative −0.0343. No tolerance
argument survives: a better feasible point demonstrably exists.
The residual is not lying. Stationarity there is 1.93e-05 absolute. §14.2 divides by max(1, |c|_inf) and this instance's |c|_inf is 2e6, so it prints 9.7e-12. A convex
problem cannot have an exactly satisfied KKT system at a suboptimal point, and this one
does not — the residual is real, it is small enough relative to data of size 1e6 that a
relative certificate cannot tell it from zero, and the conditioning turns it into a percent
error in the objective.
The certificate is relatively satisfied and the answer is wrong, and neither of those is a
mistake in the other.
Three consequences.
§6's criterion is meaningful and not sufficient. Success Criterion 2 asks for
"mathematically meaningful conic KKT residuals" and it has them. What it does not have is a
criterion that bounds the objective error on a badly scaled instance. An absolute floor, or
a normalization that does not divide by |c|_inf, is the missing piece — Conic KKT residuals and termination criterion #22's, and
now stated in the paper's assumptions rather than patched quietly.
The study was self-certifying.solved meant "COSA is satisfied with itself"; on
twelve of thirteen families that is the same thing. Success Criterion 5 asks for agreement
with a reference on every generated problem and this study was not checking. It now does,
with a fifth verdict — wrong, meaning optimal, certified and disagreeing — and unchecked when no reference is available, because "nothing disagreed with me" and "a
reference agreed with me" are different claims.
And it is the third diagnosis this family has had. It stalled (blamed on conditioning,
with equilibration apparently rescuing it); then the stall was traced to raise_free_heads
and the family ran to completion (reported solved); now it runs to completion with the wrong
answer. Each step had evidence. Two were wrong. A study that certifies itself will confirm
whichever hypothesis it started with — which is the general lesson, and it is why result 9
below is the longest entry in this list.
9. Silent failure is the default failure mode, repeatedly. Every one of these was found
by building the thing that could notice, never by inspection:
Wave 2's SCS calibration held over 200 seeds I chose and failed on one I did not. Found by a
hypothesis property test.
three more only CI could catch: a dependency floor asserted rather than resolved, a LAPACK
behaviour measured on one platform and documented as the behaviour, and a termination
tolerance set exactly at its own achievable floor.
raise_free_heads's unit-coefficient test (result 15), which made a whole family look
like a conditioning failure for two waves and had a before-and-after table agreeing with it.
A --amend after a hook-aborted commit folded a whole wave into the previous one's
commit. Caught by the next wave's PR reporting a conflict, not by anything local.
An assertion with a four-iteration margin (68 < 67 in CI, 74 < 78 locally). A claim
a different BLAS can overturn is not the claim worth making; the real effect was a third of
the work, visible in totals.
A CI timeout whose first fix did not hold. Module-scoped fixtures ordered so each test
paid for one run — void under pytest -n auto, which chooses the order. It then passed one
3.13 job and failed another at the same commit. And the local check of the second fix ran pytest -n auto with the plugin not installed, so it silently verified nothing.
A self-certified study (result 17), which is the worst of these: it did not fail
silently, it succeeded silently, for two waves, in two different directions.
The dependency graph
53 edges across 31 issues, no cycles. Every issue's earliest possible start, as a layer.
Everything within a layer is independent and can run in parallel once the earlier layers are
done.
Complete. All fourteen links, and all seventeen off-path issues with them.
Where milestone order and build order disagreed
Recorded as history now that the relevant work has landed, because the divergences were the
point of this issue.
1. M9 came before M7, and it paid immediately.#33 blocks #25 and #29 because §12.4
(paper.tex:949) says those instances exist "to identify failure modes before
optimizing performance". All of M9 except #34 and #35 landed ahead of every M5 issue but
one — and #33's degenerate-optimum family exposed that #12's singular-KKT guard never fired,
which no amount of reading would have.
2. #21 was a layer-2 issue living in M5, and paid for itself three times: it confirms all
twelve instance families feasible and bounded, it is what #20's correctness is measured
against, and building it revealed that the open reference solvers disagree by up to 1e-5.
3. M6 split across three layers, and finished four waves apart. #24 was at L4 and landed
in Wave 3; #22 was at L5 and landed in Wave 4; #23 was at L8 and landed in Wave 7. "M6" was
never a phase.
4. M7 split across four.#28 at L4, #25 at L5, #26 at L6, #27 and #29 at L9. Treating it
as a block would have blocked the prototype behind a factorization study.
What is left
One decision, and one honest piece of unfinished business.
The release tag is unblocked.#59 was the last thing between the project and a taggable
release, and it is merged: main is green on every gate, and the paper no longer contains a
claim the code contradicts. Tagging is a decision rather than a task, and it is deliberately
not automated here.
§6's criterion still has no absolute floor.#59 makes the study notice that a badly
scaled instance can terminate with a relatively-tiny residual at a percent-wrong answer; it
does not make the solver notice. That is #22's,
recorded there with the failing instance and two candidate normalizations, and stated in the
paper's §14.4 as a required assumption rather than patched quietly.
This is the right place to stop rather than a loose end: the alternative was to invent a
threshold under time pressure, and this project's record says that is exactly where the next
mistake would have gone. The study now has the instrument to judge a candidate criterion by —
a wrong verdict that has to disappear for the right reason.
All ten Final Deliverables exist and #58 names where each one is. Success Criterion 7 is demonstrated rather than asserted: tests/test_modularity.py builds SOCPs with two and
then three independent second-order cones, none of them portfolios and with overlapping
tails so the factors are not separable, and solves them through the ordinary entry point
against a reference solver. Nothing in cosa.solver is edited, subclassed or monkeypatched,
and the test asserts that of itself by parsing its own syntax tree rather than grepping its
own text.
Success criteria, honestly
criterion
verdict
1
solves the portfolio SOCPs to high accuracy
yes, on twelve of thirteen families
2
termination on mathematically meaningful conic KKT residuals
yes, and not sufficient — result 17
3
working-set decisions interpretable as active constraints
yes
4
reliable warm starts on sequences
conditionally — result 14
5
solutions agree with established SOCP solvers
no, on one family — result 17
6
numerical behaviour understood on degenerate/ill-conditioned problems
yes, including where it fails
7
modular enough for extensions beyond portfolios
yes, demonstrated
Four unqualified, three qualified, and the qualifications are the results worth having. §20
says as much: the valuable outcome is a characterization of when conic active-set methods
work well, not a claim that they always do.
#39–#42 had no milestone deliberately — they were watched throughout rather than resolved at
a point. All four are now closed with verdicts, and three of the four fired. Each issue
carries its own evidence; the short forms are below.
Worth noting before the detail: two of the four fired in a mode their own watch list did
not contain, and in both cases the watch list's proposed mitigation would not have helped.
That is the argument for tracking risks as questions rather than as predictions.
#39 — closed. Neither of the two outcomes it offered. Its condition was "either the
binary state demonstrated sufficient, or the generalization to cone faces implemented", and
what happened is neither: the state was never binary (INACTIVE/TANGENT/APEXis the
face generalization in miniature), and what proved insufficient was H = rho*I rather than
the representation. Generalizing the working set would not have fixed the thing that broke.
Residue: on eq. (7) an unjustified apex cannot be released, because dropping the factor
forces d_t = -lam/rho exactly — measured at -2.000000 against -lam/rho = -2.000000 —
which is conically infeasible by identity rather than by tolerance. blocked-at-apex on
3 of 200 randomized draws (seeds 2, 46, 156). Stated as a limitation in the paper's §14.4,
not carried as a defect.
The longer form, kept because it is the reasoning rather than the verdict: Result 2 proves the tangent-plus-exact-step construction cannot move along the
cone's boundary and result 4 proves eq. (7) cannot release an unjustified apex. #23 tested
the concept against both and the verdict is that the working-set representation was not
what was insufficient — H = rho*I was. A three-valued state (INACTIVE / TANGENT / APEX) plus a multiplier-driven deactivation rule is enough to represent the geometry;
what it could not do was step, because the direction knew only the boundary's plane.
Result 10 supplies the missing derivative and the randomized sweep goes from 162 to 192
instances reaching an optimum.
What survives is narrower and precise: on eq. (7) an unjustified apex is unreleasable by
arithmetic, and blocked-at-apex fires on 3 in 200 randomized draws. That is an apex
problem rather than a working-set problem, and it is stated as a limitation in the paper's
assumptions section rather than left as a risk.
#40 — closed. The risk fired, in a mode the watch list named but placed elsewhere. It
listed "oscillation between SOC active and inactive" and noted that hysteresis addresses it.
The cycle was that mode at the apex, where hysteresis cannot reach: the slack is exactly
zero and nothing about the geometry changes between the two states, so no threshold helps.
Result 13. §17.2's four
remedies and §8.2's hysteresis are all implemented and none of them was the fix; the cycle
was APEX ↔ INACTIVE and the remedy was to stop re-deriving a status from unchanged
geometry. Worst return count across 200 randomized instances: 486 → 2.
#41 — closed, and the residual question was not in the linear algebra. Its watch list
said "solutions that vary with problem scaling"; they do not — they are consistently wrong
either way, which is a sharper answer than the one it was looking for. The obvious route is
closed: result 6 says the KKT system does not inherit the covariance's conditioning, because
the tangent puts L in as a single row, and rank deficiency is likewise not a difficulty.
§13.3's scaling rescues no family and costs 30–80% more iterations on every one.
What is left is not in the linear algebra at all — it is in the stopping criterion.
Result 17: on a badly scaled instance the residuals certify a percent-wrong answer, and
neither robust factorization nor equilibration touches that. Also still open: result 8's
observation that pinning an apex block can leave the system conditioned at 1e18, and slow
convergence on 5 of 200 randomized instances, which is a convergence-rate question and points
at the retraction.
#42 — closed. Confirmed, predicted, and reported without flinching. Result 16: 3–30× slower than
CVXPY/Clarabel on every mode measured, overhead included. §20 anticipated this and said the
valuable result would be a characterization instead; that characterization is in the paper's
§20.1 and is the project's principal output.
This tracker named the failure mode to avoid as "drawing the wrong conclusion from a fair
loss — abandoning the approach on a benchmark it was never predicted to win, or conversely
tuning the generic benchmark until it wins". Neither happened, and the second was a live
temptation: box(8)'s 44% warm-start saving would have made a far better abstract than
"it depends on whether the active set transfers".
The one thing the plan got wrong about itself
The plan's own instruction was to build milestone-by-milestone, and this issue exists
because the dependency graph says otherwise. That was right. What the plan could not have
anticipated is how much of the work would be correcting the plan: eq. (6)'s leading
coefficient, H = rho*I, the assumption that scaling answers bad conditioning, and — result
17 — the sufficiency of a relative KKT residual were all wrong. Each was found by building
the thing that would notice, never by reading.
Six of the seventeen results above are corrections to the specification the project was
implementing, and one of them is a correction to a correction. The pattern is consistent
enough to be worth naming: every claim this project made about itself that was not
cross-checked against something external turned out to be wrong at least once. Result 17 is
the cleanest instance — a study grading its own homework, confirming two contradictory
hypotheses in succession, both times with sound evidence.
Meta-issue. Not work — the order the other 31 work packages should be resolved in, derived
from the
Depends on:graph rather than asserted.The headline: do not work milestone-by-milestone. The milestone numbers M1–M12 come
from the paper's own table (
paper.tex:1213), which is a narrative order — the sequencethe plan is best read in. The build order is different, and in several places sharply so.
Status
Everything is closed: all 31 work packages, all four risk trackers, and the correction.
This meta-issue is the last one open, and it is the record rather than the work.
COSA solves eq. (8) as a conic active-set method, warm starts along a frontier, has a public
interface, and the paper carries its results, its assumptions statement and the
characterization §20 asks for — with its one negative result stated rather than smoothed
over.
(Four issues — #29, #34, #35, #36 — stayed open for a while after their work merged, for a
duller reason worth knowing: GitHub parses
Closes #A, #Bas closing only#A. The keywordhas to repeat.)
One correction is open: #59, which closes #36.
It is the most important finding in the project and it arrived after the last wave merged — the failure
study had been certifying its own answers, and on one family the certificate is wrong. See
result 17, which is the one to read if you read only one.
The critical path is complete:
#8 → #9 → #11 → #12 → #13 → #14 → #18 → #20 → #23 → #27 → #30 → #34 → #37 → #38.#56 is closed as merged without ever reaching
main— it merged into the wave-9 branchafter
mainhad already taken that branch via #55 — so its commit rode into #57 instead,which is why #57 closed three issues. Nothing was lost.
The release tag is not tagged, and #59 is now a reason as well as a preference. Everything
the tag would certify is checked except the claim #59 corrects, and tagging a release whose
paper contains a false result would be the one irreversible mistake available here.
Both milestone moments have passed. #20 solved eq. (8) through §9 Phase III's four
ingredients; #23 replaced the tangent-only treatment with §9 Phase IV's conic working-set
logic. §16.2's five analytical checks — primal solution, active set, multipliers, SOC
activity, KKT residuals — pass on #9's hand-solved instance. Across the structured families
(
n = 8,seed = 0), with the iteration count before and after #23:H = rho*IWhat the work has established
Sixteen results. They are the reason to read this issue rather than only the milestone
table. Results 1, 2, 10, 12 and 14 reached the paper; result 15 is the one that changed the
project's own published conclusion.
1. Eq. (6) as printed is wrong — flagged for #38.
paper.tex:457gives the leadingcoefficient as
||q||^2; expanding(t + alpha*tau)^2leaves||q||^2 - tau^2. The twoagree only when
tau = 0, which is the one case eq. (7) never takes. On a direction runningalong the cone's boundary to the apex the printed formula admits a step of zero where
the true answer is
t / (-tau)— a solver using it would refuse to travel along theboundary at all. Verified against seven hand-solved intersections and 10 000 random cases.
The correction also turns the root selection from one case into three, since the leading
coefficient can vanish or go negative.
2. An iterate on the cone's boundary cannot move — this is Risk 1, and it is real.
(Answered by #23; see result 10.) Two facts, both proved and both about eq. (7):
Tangency kills eq. (6)'s middle coefficient, feasibility kills its constant term, and the
leading one is
||q||^2 - (u'q)^2, which Cauchy–Schwarz puts at or above zero;tdown, by exactlylam / rho, becausetappears in no linear row.So a linear direction plus an exact conic step cannot move off the boundary, either way.
That is the curvature of the cone, not a defect in the ratio test, and it is §3.3's warning
about becoming "an SQP method with an SOC constraint" and #39 arriving together. #20 answered
with the retraction §3.3 sanctions for the prototype — tangent step, then restore the cone by
raising the free head. #23's answer is result 10: the retraction is unavoidable for any method
holding its iterates exactly on a curved boundary, but the tangent plane was the wrong thing
to step along, and giving the direction the boundary's curvature is what makes the retraction
cheap instead of the bottleneck.
3. An active cone's multiplier has exactly one degree of freedom. At a nonzero boundary
point, conic complementarity forces
w = w_0 * (1, -u)— the tangent covector is the dualdirection. So
w in Qcollapses to the scalarnu_cone <= 0, and an active cone costs thedual side no more than an inequality does. #23's deactivation test is a statement about that
single number — and result 11 is what that number turns out to be.
4. At the apex nothing collapses, and eq. (7) can never leave it. The tangent cone of
QbecomesQitself and the normal cone becomes-Q, so the dual test is a conemembership test — §8.1's "different tangent and normal geometry", precisely. Worse: on
eq. (7) an unjustified apex can never be released, because dropping the factor forces
d_t = -lam/rho < 0and the released direction is infeasible by arithmetic.blocked-at-apexis a real solver status that fires on real draws.
5.
rhoinH = rho*Iis not a tuning parameter. The multipliers are exactlyrho-invariant and the direction scales as1/rho. Do not confuse it with §8.3's degeneracyregularization (#25), which perturbs the problem and whose cost is visible: at a stationary
point the true direction is exactly zero while the regularized one is
O(delta).6. The KKT system does not inherit the covariance's conditioning.
cond(Sigma) = 1e10and the assembled KKT matrix sits at about 10, because the tangent representation puts
Linto the system as a single row and one row has no spectrum. A genuine difference from
interior-point methods, and the kind of result §17 (
paper.tex:1345) asks for. It also means#28's scaling earns its keep on unit mismatch rather than on ill-conditioned covariances —
2e14 → 10there.7. The KKT matrix is not sparse, and cannot be made so. However sparse
Ais, the conecontributes the tangent row
g_0 - u'L, anduis a normalized covariance direction — densein every asset. One dense row and its transpose put a dense cross in the matrix. So the
sparsity a sparse
LDL^Twould exploit is destroyed by the structure that makes the problemconic. #27 should read this before optimizing. The comparison of #26 found
range-space3.25× the reference for a structural reason instead: it solves an
m-by-msystem wherethe others solve
(n + m)-by-(n + m).8.
large's optimum is at the apex, and nobody designed it that way. A rank-kcovariance over
n >> kassets leavesLwith an(n-k)-dimensional null space, so theminimum-risk portfolio has risk exactly zero and
lam > 0takes it. The family is an apexinstance for free — and pinning the apex block with most bounds active leaves
Wsquare andthe KKT system conditioned around
1e18.10.
H = rho*Iwas leaving a derivative on the table, and it was the expensive one.§4.2 writes the direction subproblem's matrix as
rho*I, and the SOCP objective is linear,so it is tempting to conclude the Lagrangian has no curvature. It does — the cone is
curved. Writing the active factor as
g(z) = ||s_1|| - s_0 <= 0, the Lagrangian's Hessian isrho*I + sum_j mu_j * G_j' * grad^2 g_j * G_j, withgrad^2 g = (I - u u')/||s_1||in thetail block. Two things make it free to use: it is positive semidefinite (because
gisconvex) so the subproblem stays uniquely solvable, and
mu_jis just the head ofw, acomponent of a multiplier #13 already computes.
That is what "primal-dual" buys, and the phrase is not decoration — the dual variable
determines the primal step. Measured:
H = rho*I(Wave 6)The multipliers used are the previous iteration's, so the scheme is a fixed point rather than
an implicit system, and the first iteration uses zero — a #23 solve and a Wave 6 solve begin
bit-for-bit identically, which is what makes every difference between them attributable.
#38 should carry this: the paper writes
H = rho*Iand, on the evidence, should not.11. On eq. (7), §7.4's deactivation rule provably never fires.
tappears in exactly onelinear row — the cone's head — and in the objective with coefficient
lam. Reading thedirection subproblem's stationarity in the
tslot givesw_0 = -nu_cone = lam + rho * d_t,which at a stationary point is
lam. Result 3 then forces||w_tail|| = |w_0|exactly. Sowsits precisely on the boundary ofQwith a strictly positive head: neither outsideQnor zero. An active cone at a mean–standard-deviation optimum always contributes a genuine
active normal.
This cuts two ways. It says the rule is correctly inert here rather than absent — away from a
stationary point
w_0is free to go negative, and #23 exercises the release on a constructedSOCP that eq. (7) cannot produce. And it removes the cycling hazard a geometric
deactivation rule would have had on this problem: there is nothing for the rule to release,
so there is nothing for §7.3 to re-acquire. #29 should know this before designing
hysteresis — the oscillation §8.2 warns about cannot be a cone activate/deactivate cycle on
eq. (7), so whatever #40's instance shows must be something else.
12. Factorization reuse works on its own metric and loses on wall clock, for a structural
reason. The share of KKT solves needing a fresh factorization falls from 98.9% to 1.4%
across the families and 98.7% to 1.5% across the randomized sweep; a whole solve typically
factorizes twice. Wall clock goes the other way: a column update against a full
(n,n)orthogonal factor costs
O(n^2)while a fresh QR of an(n,m)matrix costsO(n m^2), soupdating wins only once
m^2exceedsn. The classical argument for factorizationupdating quietly assumes a working set comparable in size to the problem, and an
active-set method on a portfolio runs with
mwell belown. Measured: 0.98× atn = 300,1.12× at
n = 500.13. The cycle that existed was not the one anyone was watching for. §17.2 names four
remedies and §8.2 names hysteresis; none of them was the problem. The revisit counter had
been counting iterations held on a working set rather than returns to it, which reported
nine hundred revisits for a solve that was merely slow and hid a genuine two-state cycle
underneath. Corrected, the worst randomized instance alternated
APEXandINACTIVE486times: #24's branch releases an apex it cannot justify, the released direction cannot be
travelled, the step moves nothing, and §7.3 reads the same unchanged geometry and puts the
factor straight back. The fix is one line of principle — do not re-derive a status from
geometry that has not changed — and the worst return count across 200 instances went to 2.
14. Warm starting is a conditional, and reporting it as a number would have hidden it.
The per-group figures are nearly identical across instances; what differs is the mix.
Correcting a belief costs more than acquiring one — a cold solve discovers the active set on
the way in, a warm one must undo a wrong answer first and then discover it anyway. So the
condition is not "the problems are close" but "the combinatorial state is the same",
and those are different conditions: a frontier can move steadily in
lamwhile its activeset jumps.
15. The failure-mode study's central finding was wrong for two waves, with an ablation
apparently confirming it. Superseded by result 17 — the conclusion below ("solves to
1e-11 unaided") is itself wrong, and for the same underlying reason. Kept because the
sequence is the finding.
badly scaledstalled at5e-2. Ruiz equilibration converted the stall into an optimum at9.7e-7. That is a clean before-and-after, it pointed at conditioning, and equilibrationbecame the family's mitigation and briefly the public interface's default. Then #37 exposed
the real cause:
raise_free_headsrequired a cone head row to select its variable with acoefficient of exactly one —
— which nothing needs, since the head is
coefficient * t + hand solvinghead >= ||tail||is one division. No rescaled instance satisfies it, so
_heads_are_freereportedFalse,the retraction was silently unavailable, and an iterate on the cone's boundary could not
move. That is result 2 arriving with its remedy switched off, not a conditioning failure.
With the restriction removed the family runs to completion with residuals under
1e-11—which the study then reported as solved. It is not solved; see result 17. What does survive
from here: §13.3's scaling rescues no family and costs 30–80% more iterations on every one,
and equilibration was perturbing the head row's coefficient rather than the conditioning.
A "diagnosed" stop is a hypothesis, not a conclusion. The ablation was sound; what it
measured was a coincidence.
16. COSA is 3–30× slower than CVXPY/Clarabel in wall clock, on every mode measured.
CVXPY spends most of its time building a problem rather than solving one, so the comparison
ought to flatter COSA and does not. Reported rather than buried — §20 asks for a
characterization of when conic active-set methods work well, which is not a claim that
they always do. Where the numbers are good is iteration and factorization counts on
sequences.
17. A relative KKT residual certified an answer that is 3.4% wrong, and this is the most
important result in the list.
badly scaledterminates reportingoptimalwith all five of §6's residuals between1e-11and
1e-15, at an objective of −0.00524. Clarabel returns −0.03956, and that point isfeasible for COSA's own feasibility check to
1e-11, with a strictly better objective. Thedirection between them is a feasible descent direction with derivative −0.0343. No tolerance
argument survives: a better feasible point demonstrably exists.
The residual is not lying. Stationarity there is
1.93e-05absolute. §14.2 divides bymax(1, |c|_inf)and this instance's|c|_infis2e6, so it prints9.7e-12. A convexproblem cannot have an exactly satisfied KKT system at a suboptimal point, and this one
does not — the residual is real, it is small enough relative to data of size
1e6that arelative certificate cannot tell it from zero, and the conditioning turns it into a percent
error in the objective.
Three consequences.
"mathematically meaningful conic KKT residuals" and it has them. What it does not have is a
criterion that bounds the objective error on a badly scaled instance. An absolute floor, or
a normalization that does not divide by
|c|_inf, is the missing piece — Conic KKT residuals and termination criterion #22's, andnow stated in the paper's assumptions rather than patched quietly.
solvedmeant "COSA is satisfied with itself"; ontwelve of thirteen families that is the same thing. Success Criterion 5 asks for agreement
with a reference on every generated problem and this study was not checking. It now does,
with a fifth verdict —
wrong, meaning optimal, certified and disagreeing — anduncheckedwhen no reference is available, because "nothing disagreed with me" and "areference agreed with me" are different claims.
#59 corrects it. Verdicts go from 39 solved to
36 solved, 3 wrong.
And it is the third diagnosis this family has had. It stalled (blamed on conditioning,
with equilibration apparently rescuing it); then the stall was traced to
raise_free_headsand the family ran to completion (reported solved); now it runs to completion with the wrong
answer. Each step had evidence. Two were wrong. A study that certifies itself will confirm
whichever hypothesis it started with — which is the general lesson, and it is why result 9
below is the longest entry in this list.
9. Silent failure is the default failure mode, repeatedly. Every one of these was found
by building the thing that could notice, never by inspection:
dependent set gives
1e-18. Found by Six robustness instance families #33's degenerate family, within minutes of it existing.the basic family's.
nearly_redundantperturbed a row's magnitude, leaving the pair exactly parallelrather than nearly so. Found by Rank detection, regularization and dependent-constraint removal #25's rank detection disagreeing with the docstring.
§14.1's per-iterate invariant checker — which is why Solver instrumentation and per-iterate invariant checker #15 was scheduled early.
hypothesis property test.
behaviour measured on one platform and documented as the behaviour, and a termination
tolerance set exactly at its own achievable floor.
Metrics.runtimewas zero for every solve ever recorded.solvebuilds itsSolutionfrom inside the
with recorder.solving()block, so a runtime latched in that block'sfinallywas not yet set when anybody read it. Wrong since Solver instrumentation and per-iterate invariant checker #15; Four-mode comparison study with accuracy and performance tables #34 was the first consumerwith a reason to look. Peak memory had the same shape of bug.
raise_free_heads's unit-coefficient test (result 15), which made a whole family looklike a conditioning failure for two waves and had a before-and-after table agreeing with it.
--amendafter a hook-aborted commit folded a whole wave into the previous one'scommit. Caught by the next wave's PR reporting a conflict, not by anything local.
68 < 67in CI,74 < 78locally). A claima different BLAS can overturn is not the claim worth making; the real effect was a third of
the work, visible in totals.
paid for one run — void under
pytest -n auto, which chooses the order. It then passed one3.13 job and failed another at the same commit. And the local check of the second fix ran
pytest -n autowith the plugin not installed, so it silently verified nothing.silently, it succeeded silently, for two waves, in two different directions.
The dependency graph
53 edges across 31 issues, no cycles. Every issue's earliest possible start, as a layer.
Everything within a layer is independent and can run in parallel once the earlier layers are
done.
#8Scaffold the six packages and pin the numerical stack✅#9SOCP problem representation, cone spec and sign convention✅#10Mean-standard-deviation portfolio to SOCP builder✅#11Working-set representation and add/drop operations✅#16SOC predicates: membership, boundary and apex✅#21Reference-solver adapter with an open CI fallback✅#12KKT assembly and solve for the direction subproblem✅#17SOC tangent condition and normal at a boundary point✅#19Basic and box-constrained portfolio generators✅#31Sector, factor-exposure, turnover and large-instance generators✅#32Seeded random problem generator and property-based tests✅#13Multiplier computation and sign tests✅#15Solver instrumentation and per-iterate invariant checker✅#24Apex branch at Lx=0 via normal-cone conditions✅#28Scaling across the five named targets✅#33Six robustness instance families✅#14Phase I loop: initialization, ratio test, termination✅#22Conic KKT residuals and termination criterion✅#25Rank detection, regularization and dependent-constraint removal✅#18Exact SOC step interval and the conic ratio test✅#26Factorization strategy comparison against the M2 reference✅#20COSA prototype: polyhedral active set with SOC tangent✅#23Primal-dual conic working-set logic and SOC deactivation✅#27Factorization reuse, rank-one updates and SOC-tangent updates✅#29Anti-cycling rules and SOC tolerance hysteresis✅#30Warm-start API reusing solution, working set and factorizations✅#36Failure-mode and degeneracy study✅#34Four-mode comparison study with accuracy and performance tables✅#35Efficient-frontier warm-start experiment✅#37Public portfolio interface, docs and reproducible experiments✅#38Paper results, assumptions statement and release✅Critical path
14 of the 31 issues form the longest chain. It is the project's floor: no amount of
parallelism finishes sooner than this sequence.
Complete. All fourteen links, and all seventeen off-path issues with them.
Where milestone order and build order disagreed
Recorded as history now that the relevant work has landed, because the divergences were the
point of this issue.
1. M9 came before M7, and it paid immediately. #33 blocks #25 and #29 because §12.4
(
paper.tex:949) says those instances exist "to identify failure modes beforeoptimizing performance". All of M9 except #34 and #35 landed ahead of every M5 issue but
one — and #33's degenerate-optimum family exposed that #12's singular-KKT guard never fired,
which no amount of reading would have.
2. #21 was a layer-2 issue living in M5, and paid for itself three times: it confirms all
twelve instance families feasible and bounded, it is what #20's correctness is measured
against, and building it revealed that the open reference solvers disagree by up to
1e-5.3. M6 split across three layers, and finished four waves apart. #24 was at L4 and landed
in Wave 3; #22 was at L5 and landed in Wave 4; #23 was at L8 and landed in Wave 7. "M6" was
never a phase.
4. M7 split across four. #28 at L4, #25 at L5, #26 at L6, #27 and #29 at L9. Treating it
as a block would have blocked the prototype behind a factorization study.
What is left
One decision, and one honest piece of unfinished business.
The release tag is unblocked. #59 was the last thing between the project and a taggable
release, and it is merged:
mainis green on every gate, and the paper no longer contains aclaim the code contradicts. Tagging is a decision rather than a task, and it is deliberately
not automated here.
§6's criterion still has no absolute floor. #59 makes the study notice that a badly
scaled instance can terminate with a relatively-tiny residual at a percent-wrong answer; it
does not make the solver notice. That is #22's,
recorded there with the failing instance and two candidate normalizations, and stated in the
paper's §14.4 as a required assumption rather than patched quietly.
This is the right place to stop rather than a loose end: the alternative was to invent a
threshold under time pressure, and this project's record says that is exactly where the next
mistake would have gone. The study now has the instrument to judge a candidate criterion by —
a
wrongverdict that has to disappear for the right reason.All ten Final Deliverables exist and #58 names where each one is. Success Criterion 7 is
demonstrated rather than asserted:
tests/test_modularity.pybuilds SOCPs with two andthen three independent second-order cones, none of them portfolios and with overlapping
tails so the factors are not separable, and solves them through the ordinary entry point
against a reference solver. Nothing in
cosa.solveris edited, subclassed or monkeypatched,and the test asserts that of itself by parsing its own syntax tree rather than grepping its
own text.
Success criteria, honestly
Four unqualified, three qualified, and the qualifications are the results worth having. §20
says as much: the valuable outcome is a characterization of when conic active-set methods
work well, not a claim that they always do.
Two scheduling cautions
issue and reads as parallelisable, but it hard-blocks on Factorization reuse, rank-one updates and SOC-tangent updates #27: §9 Phase VI
(
paper.tex:778) reuses "previous KKT factorizations when possible", so a factorizationobject with a reuse API must exist first. What Warm-start API reusing solution, working set and factorizations #30 can start from today is
PortfolioInstance.witnessandfeasible_start, which already accept a supplied point andrefuse an infeasible one rather than silently ignoring it.
already earned it twice: KKT assembly and solve for the direction subproblem #12's deliberate refactorize-every-iteration policy gives its
factorization counter a reference value for Factorization reuse, rank-one updates and SOC-tangent updates #27 to beat, and its Level 1 checker caught a
step-guard bug four waves before the frontier experiment would have.
Risk trackers
#39–#42 had no milestone deliberately — they were watched throughout rather than resolved at
a point. All four are now closed with verdicts, and three of the four fired. Each issue
carries its own evidence; the short forms are below.
Worth noting before the detail: two of the four fired in a mode their own watch list did
not contain, and in both cases the watch list's proposed mitigation would not have helped.
That is the argument for tracking risks as questions rather than as predictions.
#39 — closed. Neither of the two outcomes it offered. Its condition was "either the
binary state demonstrated sufficient, or the generalization to cone faces implemented", and
what happened is neither: the state was never binary (
INACTIVE/TANGENT/APEXis theface generalization in miniature), and what proved insufficient was
H = rho*Irather thanthe representation. Generalizing the working set would not have fixed the thing that broke.
Residue: on eq. (7) an unjustified apex cannot be released, because dropping the factor
forces
d_t = -lam/rhoexactly — measured at-2.000000against-lam/rho = -2.000000—which is conically infeasible by identity rather than by tolerance.
blocked-at-apexon3 of 200 randomized draws (seeds 2, 46, 156). Stated as a limitation in the paper's §14.4,
not carried as a defect.
The longer form, kept because it is the reasoning rather than the verdict: Result 2 proves the tangent-plus-exact-step construction cannot move along the
cone's boundary and result 4 proves eq. (7) cannot release an unjustified apex. #23 tested
the concept against both and the verdict is that the working-set representation was not
what was insufficient —
H = rho*Iwas. A three-valued state (INACTIVE/TANGENT/APEX) plus a multiplier-driven deactivation rule is enough to represent the geometry;what it could not do was step, because the direction knew only the boundary's plane.
Result 10 supplies the missing derivative and the randomized sweep goes from 162 to 192
instances reaching an optimum.
What survives is narrower and precise: on eq. (7) an unjustified apex is unreleasable by
arithmetic, and
blocked-at-apexfires on 3 in 200 randomized draws. That is an apexproblem rather than a working-set problem, and it is stated as a limitation in the paper's
assumptions section rather than left as a risk.
#40 — closed. The risk fired, in a mode the watch list named but placed elsewhere. It
listed "oscillation between SOC active and inactive" and noted that hysteresis addresses it.
The cycle was that mode at the apex, where hysteresis cannot reach: the slack is exactly
zero and nothing about the geometry changes between the two states, so no threshold helps.
Result 13. §17.2's four
remedies and §8.2's hysteresis are all implemented and none of them was the fix; the cycle
was
APEX↔INACTIVEand the remedy was to stop re-deriving a status from unchangedgeometry. Worst return count across 200 randomized instances: 486 → 2.
#41 — closed, and the residual question was not in the linear algebra. Its watch list
said "solutions that vary with problem scaling"; they do not — they are consistently wrong
either way, which is a sharper answer than the one it was looking for. The obvious route is
closed: result 6 says the KKT system does not inherit the covariance's conditioning, because
the tangent puts
Lin as a single row, and rank deficiency is likewise not a difficulty.§13.3's scaling rescues no family and costs 30–80% more iterations on every one.
What is left is not in the linear algebra at all — it is in the stopping criterion.
Result 17: on a badly scaled instance the residuals certify a percent-wrong answer, and
neither robust factorization nor equilibration touches that. Also still open: result 8's
observation that pinning an apex block can leave the system conditioned at
1e18, and slowconvergence on 5 of 200 randomized instances, which is a convergence-rate question and points
at the retraction.
#42 — closed. Confirmed, predicted, and reported without flinching. Result 16: 3–30× slower than
CVXPY/Clarabel on every mode measured, overhead included. §20 anticipated this and said the
valuable result would be a characterization instead; that characterization is in the paper's
§20.1 and is the project's principal output.
This tracker named the failure mode to avoid as "drawing the wrong conclusion from a fair
loss — abandoning the approach on a benchmark it was never predicted to win, or conversely
tuning the generic benchmark until it wins". Neither happened, and the second was a live
temptation:
box(8)'s 44% warm-start saving would have made a far better abstract than"it depends on whether the active set transfers".
The one thing the plan got wrong about itself
The plan's own instruction was to build milestone-by-milestone, and this issue exists
because the dependency graph says otherwise. That was right. What the plan could not have
anticipated is how much of the work would be correcting the plan: eq. (6)'s leading
coefficient,
H = rho*I, the assumption that scaling answers bad conditioning, and — result17 — the sufficiency of a relative KKT residual were all wrong. Each was found by building
the thing that would notice, never by reading.
Six of the seventeen results above are corrections to the specification the project was
implementing, and one of them is a correction to a correction. The pattern is consistent
enough to be worth naming: every claim this project made about itself that was not
cross-checked against something external turned out to be wrong at least once. Result 17 is
the cleanest instance — a study grading its own homework, confirming two contradictory
hypotheses in succession, both times with sound evidence.