[2020] inconsistencies in new filter syntaxes

Originally posted by Xeverous

/u/RhysGGG, can you clarify this? And /u/_xhul_, FYI:

https://cdn.discordapp.com/attachments/175290321695932416/678285728093831177/unknown.png

• I got the post on == vs = but:

• I'm unsure whether the syntax is SocketGroup "GGG" or SocketGroup GGG. There are 2 conflicting examples (in red).

• I'm also unsure how both Sockets and SocketGroup will behave when there are multiple color groups (with = and == we have 4 different cases)

• It's also not specified whether the order of letters matters - you never published any EBNF spec or anything similar and the parser container multiple unintuitive/bugged assumptions previously so I'm asking if there is anything regarding syntax that is worth to know

• It's unclear whether CorruptedMods will work with comparison operators. Also why only 0-2 range? What If I drop the multi-corrupted Sirus gloves?

Originally posted by Dissolator

Please check if those are correct:

SocketGroup == 5RRR

shall fail in any case since it needs 5 links and exactly "RRR" group, which is 3 link, so it doesn't match

Sockets == 5RRR

will work and catch any 5 sockets (may not be linked) with 3x Red sockets in any order, since "RRR" here is not a socket group but just a condition to have number of specific types(colors)... or "==" will require number of sockets to be matched to number of listed socket types?

Originally posted by Dissolator

Thanks! So exact match "==" with "5RRR" will not match if item has 4 red sockets, but match item with 3 red and 2 sockets of any other color?

what if it's SocketGroup == 5RRG - it'll match 2 red 1 green and others are not red and not green? while regular match = will allow at least 2 red and 1 green and others may be red or green too?

How < will work with "4RG" ? It will match 3 or 2 linked items with at least 1 red and 1 green in the link? so "RG" part does regular "=" check, not "less then", otherwise it hard to get what "less than 1 red and 1 green" shall check. So < <= => > operators are only applied to number of links, and socket types are have regular "=" match in all those cases?

Originally posted by _xhul_

I wasn't expecting that.

Are you implying that "SocketGroup < 4RG" is parsed the same way as "SocketGroup < 4 R G"?

Originally posted by Xeverous

I have another question. The post mentions "By default (operator =) the number requires an exact match." but gives examples using only ==.

So:

• What is the difference between SocketGroup == 5GGG and SocketGroup = 5GGG?
• What is the difference between Sockets == 5GGG and Sockets = 5GGG?

Originally posted by Xeverous

I have another questions:

• What is the difference between = and == for AreaLevel condition (the official post mentions both)? So far all such conditions (ItemLevel, DropLevel, Quality, GemLevel etc) only allowed < or > or <= or >= or = but not ==.
• What is the default opacity value for colors that do not specify it? I noticed it is not fully opaque.

Originally posted by Xeverous

Thanks, but didn't you mean (255 - 50)? 0 is fully transparent, 255 is fully opaque. Default black-but-a-bit-transparent background for common items looks more like 205 than 50.

Originally posted by Xeverous

Thanks, do you mind checking 2 more things? /u/_xhul_ mentioned few really weird things in https://github.com/xhul-github/xhul-filter/blob/master/misc.txt - I'm mostly concerned about legacy prophecies that are not accepted and == being reported to not perform an exact string-string match for Class as it is the case for BaseType.

Originally posted by _xhul_

Thanks a lot for those precisions.

I retested another bug i mentioned previously, and since you didn't reply, just in case you missed it: The 3.9.2f recoding of "Sockets" & "SocketGroup" has a flaw, specifically when used with a "0" argument. A few examples to illustrate: "Sockets 0" and "SocketGroup 0" will never match an Iron Ring, despite it has no sockets. Similarly, "Sockets 0" and "SocketGroup 0" will always match an Unset Ring, despite it actually has 1 socket. It's worth noting that that behaviour seems to be restricted to non-weapon & non-armour classes only ("Sockets 0" and "SocketGroup 0" will successfully match Kaom's Roots, for example).

Glad to hear you'll take a look at the interaction between "Class" & "==", that will actually grant us more flexibility in the order in which we declare our blocks, cool.

About legacy prophecies, i was surprised they got that treatment, compared to the many player-drop-only bases that can still be referred to, i guess because the "Prophecy" condition was implemented after those got removed from the pool of seekable ones.

Originally posted by _xhul_

I was partly wrong, the bug isn't actually specific to "0" arguments, more like specific to non-weapon and non-armour classes in their intirety. For example, "Sockets < 6" and "SocketGroup < 6" fail to match an Iron Ring aswell.

While i'm here, could we get some details about that "Added new item filter drop effect colour options." 3.10 patch note?

Originally posted by Xeverous

Isn't there a sort of distinction between matching against 0 and "null"?

For example, Class "Divination Card" will match any card. But If I add Sockets <= 6 it will stop matching any card because cards are not treated to have 0 sockets, they are treated to have "null" sockets and any comparison requiring any socket number always fails.

Is such behavior intended? Can you list what items have "null" instead of 0? /u/_xhul_ already mentioned some inconsistency for sockets with jewellery items that seems to exhibit such implementation.

Originally posted by briansd9

Hi /u/RhysGGG, thank you for the clarifications, I have another question.

Resonator sockets are currently indicated by "DV" in the JSON returned from the item data API. Will this be changed to "D" to conform with the filter syntax?

Originally posted by Xeverous

For sockets specifically, base types that could never have sockets were matching false against all socket filters, even things like Sockets <= 6

That's exactly how comparison against null or NaN works - it's always false.

I have changed this in 3.10.0 so everything has 0 sockets, and will match as such.

How about other properties? Eg if I do Class "Divination Card" MapTier >= 1 will it catch cards or not? If not, then it (possibly accidentally) implements "comparison against null/nan" behavior. I'm just interested which conditions have such behavior.

Originally posted by Xeverous

Hello again, during implementing own filter simulator/debugger I once more reviewed this thread and performed some experiments. Some results were different than I expected.

My goal is to understand exactly how the Sockets and SocketGroup conditions work.

---

First, I tested all the examples here and whether their wording is correct. Everything is fine here, but I post for clarity:

• SocketGroup 5GGG - an item with exactly 5 linked sockets AND at least 3x G linked within such group
• SocketGroup = 5GGG - an item with exactly 5 linked sockets AND at least 3x G linked within such group
• SocketGroup == 5GGG - an item with exactly 5 linked sockets AND exactly 3x G linked within such group
• SocketGroup >= 5GGG - an item with at least 5 linked sockets AND at least 3x G linked within such group
• Sockets 5GGG - an item with exactly 5 sockets AND at least 3x G socket
• Sockets = 5GGG - an item with exactly 5 sockets AND at least 3x G socket
• Sockets == 5GGG - an item with exactly 5 sockets AND exactly 3x G socket
• Sockets >= 5GGG - an item with at least 5 sockets AND at least 3x G socket

5s 5L 3G   5L 2G + 1G     5s 5L 4G     5L 3G + 1G      6L 3G     6L 4G
  +            -             +              +            -         -
  +            -             +              +            -         -
  +            -             -              +            -         -
  +            -             +              +            +         +
  +            -             +              -            -         -
  +            -             +              -            -         -
  +            -             -              -            -         -
  +            +             +              +            +         +

---

Then I tested less-than comparisons and one result is different than expected (marked with | |)

• SocketGroup < 4RG - an item with less than a 4-link OR less than 1x R socket OR less than 1x G socket
• Sockets < 4RG - an item with less than 4 sockets OR less than 1x R socket OR less than 1x G socket

B-B-B   G-G-G-G   R-R-R-R   R-G-B-B   R-G B-B   B-B-B-B R G
  +        +         +         -         +          |+|
  +        +         +         -         -           -

---

Then I tested more-than comparisons:

• SocketGroup > 4RG - an item with more than a 4-link OR more than 1x R socket OR more than 1x G socket
• Sockets > 4RG - an item with more than 4 sockets OR more than 1x R socket OR more than 1x G socket

B-B-B-B-B   R-R   R R   G-G   G G   R-G   R-G-B-B   R-G-B-B B
    +        +    |-|    +    |-|    -       -          -
    +        +     +     +     +     -       -          +

---

To summarize, the only thing I don't get is how/when SocketGroup counts colors when they are not linked together. Can you correct the wording of these or give me any sort of algorithm how these are implemented?

• SocketGroup < 4RG - an item with less than a 4-link OR less than 1x R socket OR less than 1x G socket
• SocketGroup > 4RG - an item with more than a 4-link OR more than 1x R socket OR more than 1x G socket

My intuition is:

• SocketGroup < 4RG - an item with less than a 4-link OR less than 1x R socket within any linked-sockets-group OR less than 1x G socket within any linked-sockets-group
• SocketGroup > 4RG - an item with more than a 4-link OR more than 1x R socket within any linked-sockets-group OR more than 1x G socket within any linked-sockets-group

...where I define linked-sockets-group as a group of sockets tied by links (so item R-G-B B B-R has 3 groups: RGB, B, BR).

Thanks for any response and have a good weekend.

Originally posted by Xeverous

Thanks for the response. So in short, SocketGroup performs matches per each linked-socket-group (at least 1 group needs to satisfy) and Sockets performs one match against item total sockets and total color counts. This is really good news, because I thought of many possible implementations of representing item sockets and eventually went with an array of link-groups, which now turned out to very closely related to how SocketGroup works.

Let me preface by saying I feel like the < and > operators are generally not super useful or intuitive for socket matching, but I implemented them anyway for consistency...

IMO this condition itself is really complex and rarely used but anything more intuitive would surely require much more complex implementation.

Are the sockes/links conditions on https://www.pathofexile.com/trade working the same way? Haven't analyzed them thoroughly.

For the > operator, your intuition looks correct. Though considering it again, I somewhat regret implementing > like that, because it's inconsistent with >= semantically (ie an item can be > but not >= which is weird).

I agree, it's a bit weird. < and > use "OR"s but <=, >=, == and = use "AND"s. Thus, satisfying <= and >= is harder than < and >.

If there are any other non-obvious issues with the filtering, please let me know. You mentioned in the data mining thread that there were some subtle things that could catch people out, but afaik any such problem should either warn or error now.

• Is the Stackable Currency for non-stackable things intended?
• I know that you use some naive-style implementation (that is, string-find-if-else instead of actual recursive grammar parsers) for parsing filters, but I haven't found any risky bugs that would cause bigger problems than allowing funny lines like CustomAlertSound WHERE IS MY HH???"pop.wav".
• Lasty, I guess there is no way to change it now but - there are multiple Item Class systems in the game. What's there reasoning behind them? Filters have their own Class names and values that do not always agree with what is reported on trade APIs or in game's item metadata.