java - Custom Email Filter in Solr doesn't work -


i have created solr filter retrieve email specific text , return email!

this code:

public final class normalizeaffliationfilter extends tokenfilter {  private chartermattribute chartermattr;    protected normalizeaffliationfilter(tokenstream ts) {      super(ts);      this.chartermattr = addattribute(chartermattribute.class);    }    @override    public boolean incrementtoken() throws ioexception {      if (!input.incrementtoken()) {        return false;      }     string token =chartermattr.tostring();        pattern pattern = pattern.compile("([a-z0-9_.-]+)@([a-z0-9_.-]+[a-z])");         matcher matcher = pattern.matcher(token);         stringbuilder sb = new stringbuilder();         while(matcher.find()){             sb.append(matcher.group());         }         sb.append("  ");      string email = sb.tostring();        chartermattr.setempty();        chartermattr.copybuffer(email.tochararray(), 0, email.length());       return true;    }

i've added field type , field in schema.xml

 <fieldtype name="emailnormalized" class="solr.textfield">   <analyzer type="query">     <tokenizer class="solr.classictokenizerfactory"/>   <filter class="ir.pandapp.normalizeaffliationfilterfactory"/>   <filter class="solr.lowercasefilterfactory"/>   </analyzer>     <analyzer type="index">     <tokenizer class="solr.classictokenizerfactory"/>   <filter class="ir.pandapp.normalizeaffliationfilterfactory"/>  <filter class="ir.pandapp.normalizeaffliationfilterfactory"/>  <filter class="solr.lowercasefilterfactory"/>   </analyzer> </fieldtype>  <field name="mods.affiliation" type="emailnormalized" indexed="true" stored="true"  multivalued="true"/>

i've added sysout in code logs , works! gets token , email token return!

i've tested on analysis in solr:

my results

after of when search in solr, doesn't work!

like if field value is:"aaaaemail:something@something.com" , search:"aaaa" returns doc!

but should return when search:"something@something.com". have checked schema browser has indexed emails(the correct form). i've got no idea check next! know missing?

no custom code required. need invert described in remove email address solr indexing

as such make use of uax29urlemailtokenizer add type meta data tokens of text , use typetokenfilter let types pass of liking. in case <email>.

alter field type emailnormalized in schema.xml follows

<fieldtype name="emailnormalized" class="solr.textfield">     <analyzer>         <tokenizer class="solr.uax29urlemailtokenizerfactory"/>         <filter class="solr.typetokenfilterfactory"                  types="email_type.txt" usewhitelist="true"/>         <filter class="solr.lowercasefilterfactory" />     </analyzer> </fieldtype>

create file named email_type.txt in conf folder, should same place schema.xml resident. file needs 1 line of content

<email>

should have trouble delimiters used tokenizer, can tweak using patternreplacecharfilter. charfilters may go before tokenizer. work sample text have in image, replacing colons blank.

<fieldtype name="emailnormalized" class="solr.textfield">     <analyzer>         <charfilter class="solr.patternreplacecharfilterfactory"                      pattern=":" replacement=" "/>         <tokenizer class="solr.uax29urlemailtokenizerfactory"/>         <filter class="solr.typetokenfilterfactory" types="email_type.txt" usewhitelist="true"/>         <filter class="solr.lowercasefilterfactory" />     </analyzer> </fieldtype>

Comments

Post a Comment

Popular posts from this blog

node.js - Node js - Trying to send POST request, but it is not loading javascript content -

i have been trying send post request website, , has not been loading full site. i guessing problem node js not loading javascript content, executing it. can see javascript in response, though never loads content javascript files producing in html. what mean is, doesn't execute javascript. how make node js code overcome this, , wait the full website load? have tried native code, , "request' module, both producing same errors. if helps, don't need native javascript. modules fine. some code have tried (request module): var options = { method: 'post', uri: 'https://url', formdata: { 'email': 'email', 'prize': '0', 'transactional': 'on' }, headers: { /* 'content-type': 'application/x-www-form-urlencoded' */ // set automatically } }; rp(options) .then(function (body) { console.log(body) }) .catch(function (err) { console.log(err) }); when fetch content...
Read more

javascript - Replicate keyboard event with html button -

i have tried few ways make happen. have javascript pinball game uses keyboard keys controls. converting application touch screen display. workaround, hoping overlay buttons when clicked simulate keyboard strokes. have done research , sounds should work. using jquery v3.2.1 here things have tried far... html element: <input type="button" class="coil"></input> $('.coil').trigger({type: 'keydown', which: 40, keycode: 40}); with this, not errors, nothing happens. var coilkey = $('body').keydown(function(e) { switch (e.keycode) { case 40: break;} }) $('.coil').click(coilkey); this 1 returns error in console... ((jquery.event.special[handleobj.origtype] || (intermediate value)).handle || handleobj.handler).apply not function below javascript game engine handles key listeners... addkeylistener: function (keyandlistener) { this.keylisteners.push(keyandlistener); }, // gi...
Read more

javascript - Web audio api 5.1 surround example not working in firefox -

i've been playing lott lately web audio api on both firefox , chrome. few days ago started experiment surround set. when trying surround audio in web audio api came notice example works fine in chrome v59 not in firefox v54. // create web audio api context var audioctx = new (window.audiocontext || window.webkitaudiocontext)(); audioctx.destination.channelinterpretation = 'discrete'; audioctx.destination.channelcountmode = 'explicit'; audioctx.destination.channelcount = 6; var oscillators = []; var merger = audioctx.createchannelmerger(6); console.log(audioctx.destination, merger); //merger.channelinterpretation = 'discrete'; //merger.channelcountmode = 'explicit'; //merger.channelcount = 6; var addoscilator = function(channel, frequency) { var oscillator = audioctx.createoscillator(); oscillator.frequency.value = frequency; // value in hertz oscillator.connect(merger,0,channel); oscillator.start(); oscillators.push(o...
Read more