This is a BETA. I will NOT BE RESPONSIBLE for any damages this may cause in your repository. It is not my intention to write a super-in-detail guide, just a quick cheatsheet for programmers who know what they are doing
this post has been updated due to a mail asking how to implement autocomplete in textarea with multiple lines (multiple advisors). Refer to the last section to see changes.
Updated again 2011-05-24 for NOT SHOWING VALUES FROM DELETED RECORDS. Refer to last section
Updated again 2011-05-27 considering utf8 issues. Refer to last section
Updated again 2011-06-06 for multiple field autocomplete. Click here for the last version of code
It’s been a long time me wanting to have some authority control in our CDS Invenio repository. Neither the current version nor the 1.0 Candidate release are implementing this. So I got hands on job. We will be using jQuery and PHP (but this could be easily done in python too)
The problem
Our users were submiting theses records. The advisors (those who “coach” the students in their thesis research) tend to repeat, but the students keep introducing their names in a different way. For instance, I noticed that the same advisor (Montijano, J.I.) was named in three ways: (1) Montijano, J.I. ; (2)Montijano, Juan I. ; (3) Montijano, Juan Ignacio
As a computer engineer in a librarian environment… this was irritating me: I wanted the advisor names to be consistent. Because of that I am demonstrating how implement an autosuggest (aka autocomplete) for the submitting form, so that when the student begins to write “Monti” a box with “Montijano, J.I.” shows up, he clicks it, and the textarea field is filled with that value. Less typos, more consistency. Cool!
The fix
First, make sure that your site is already loading jQuery lib. Mine does (in an special-optimized way):
<script src="https://zaguan.unizar.es/combine.php?type=javascript&files=jquery.js,spoiler.js" type="text/javascript"><!--mce:0--></script> |
The textarea field in the submitting form
Then we will create the textarea element we want to autocomplete. Mine is called DEMOTHE_SUPERV. Please notice that the value of this field will be stored in field ’700__a’
The Javascript (jQuery)
Then we are going to create a new element description called DEMOTHE_AUTOC (type=response) wich will include the javascript code (and the CSS styling lines). There are a lot of comments in the code to help you understand what each part does:
text=''' <script type="text/javascript"><!--mce:1--></script> <!-- now the CSS3 part to make the suggestion box stylish --> <!-- .suggestionsBox { position: relative; left: 30px; margin: 10px 0px 0px 0px; width: 200px; background-color: #212427; -moz-border-radius: 7px; -webkit-border-radius: 7px; border: 2px solid #000; color: #fff; } .suggestionList { margin: 0px; padding: 0px; } .suggestionList li { margin: 0px 0px 3px 0px; padding: 3px; cursor: pointer; } .suggestionList li:hover { background-color: #659CD8; } --> <!-- and now an empty div, which initially is NOT displayed (don't show any suggestions when there are not suggestions LOL) --> <div id="suggestions" class="suggestionsBox" style="display: none;"> <img style="position: relative; top: -12px; left: 30px;" src="/preview/upArrow.png" alt="upArrow" /> <div id="autoSuggestionsList" class="suggestionList"></div> </div> ''' |
Also copy upArrow.png file to /preview/upArrow.png path in your apache server.
The PHP program to retrieve database results
Now, the PHP part (which should be accesible in the /preview/autocomplete.php path in your server!)
<!--?php //works with php5 + mysqli // EDIT HERE!! $db = new mysqli('localhost', 'yourUsername', 'yourPassword', 'yourDatabase'); if(!$db) { // Show error if we cannot connect. echo 'ERROR: Could not connect to database'; } else { // Is there a post'ed query string? if(isset($_POST['queryString'])) { $queryString = $db--->real_escape_string($_POST['queryString']); // Is the string length greater than 0? if(strlen($queryString) >0) { // Run the query: We use LIKE '$queryString%' // Spanish and other UTF8 users NEED TO SET THIS!!!! $query = $db->query("SET NAMES 'utf8'"); // YOU NEED TO ALTER THE QUERY TO MATCH YOUR DATABASE!! Mine is stored in bib70x table. Where is yours? /* this commented query will make that the stored tags corresponding to DELETED records will also show up */ /* the second query (uncommented) filters these so that they do not show! */ /*$query = "SELECT distinct value FROM cdsinvenio.bib70x WHERE value LIKE '$queryString%' and tag='700__a' LIMIT 15";*/ $query = "SELECT distinct value FROM cdsinvenio.bib70x,cdsinvenio.bibrec_bib70x WHERE bib70x.value LIKE '$queryString%' and tag='700__a' and bib70x.id = bibrec_bib70x.id_bibxxx LIMIT 15"; $query = $db->query($query); if($query) { // While there are results loop through them - fetching an Object. while ($result = $query ->fetch_object()) { // Format the results, im using <li> for the list, you can change it. // The onClick function fills the textbox with the result. echo '</li> <li onclick="fill(\''.$result->value.'\');">'.$result->value.'</li> '; } } else { echo 'ERROR: Issues found retrieving results :-('; } } else { // Dont do anything. } // There is a queryString. } else { echo 'You should not be calling this from your browser!'; } } ?> |
Screenshot
This is a screenshot of the working beta. Not very stylish now, but working like a charm!
Security concerns
This implementation is a BETA. It produces severe mysql load. You should not use this if there are lots of users submitting records at the same time. You should also take the necessary steps to protect your mysql server from a DoS attack. You should also grant privileges to the mysql ‘user’ from only the ‘hostname’ which hosts the autocompletar.php file.
Other concerns
When this begins to work, you will notice A LOT of inconsistencies in your database. The ’700__a’ values corresponding to deleted records will show up (note the changed query in the php program!!!) The inconsistencies in names (as the one I mentioned earlier of Montijano, J.I.) will be showing up.
You will have to ALTER your database in order to fix this and not have your users confused. I am still guessing if altering the bib70x table in the database has any uncontrolled side effects on the behaviour of invenio. I will be grateful if anyone already knows this and can help me.
—- UPDATED 2011-05-24 —-
When retrieving results from bib70x table I find a lot of values (for instance, ‘id=3108,tag=’700__a’,value=’AGUDO’) which do not correspond to any “live” record (they might have been in the past, in records which now are deleted).
I noticed tag values corresponding with deleted records were still showing up in autocomplete box. Ugly. Tibor (from CDS Support Team) suggested that I might want to run:
sudo -u apache inveniogc -b |
Well, I did run it. Did not make what I wanted.
BUT: Some time ago I showed some CDSInvenio mysql queries that I found useful for instance, to list deleted records (refer to the link for in-depth explanations):
SELECT DISTINCT bibrec_bib98x.id_bibrec FROM bibrec_bib98x WHERE id_bibxxx=3 |
So, why not use this to achieve what we want? I know this subquery is not super fast (if a thousand submitters in the same form at the same time… shit would happen), but it’ll do the job for our library. Or so I hope.
So, edit your autocomplete.php and change the query to:
$query = "SELECT distinct value FROM cdsinvenio.bib70x,cdsinvenio.bibrec_bib70x WHERE bib70x.value LIKE '$queryString%' and tag='700__a' and bib70x.id = bibrec_bib70x.id_bibxxx AND bibrec_bib70x.id_bibrec NOT IN (SELECT DISTINCT bibrec_bib98x.id_bibrec from bibrec_bib98x WHERE id_bibxxx=3) LIMIT 15"; |
——END OF UPDATE 2011-05-24————
I was not aware of distinct behaviour regarding utf8 strings with accents. For further details, refer to this link
The query for showing both accented nor accented strings is like follows:
$query = "SELECT DISTINCT BINARY value,value FROM cdsinvenio.bib70x,cdsinvenio.bibrec_bib70x WHERE bib70x.value LIKE '$queryString%' and tag='700__a' and bib70x.id = bibrec_bib70x.id_bibxxx AND bibrec_bib70x.id_bibrec NOT IN (SELECT DISTINCT bibrec_bib98x.id_bibrec from bibrec_bib98x WHERE id_bibxxx=3) LIMIT 15"; |
——– UPDATE 2011-05-27—————
——END OF UPDATE 2011-05-27————
Please feel free to use the contact form in the top menu for suggestions, but I’ll prefer if you post em here, in the comments section, so that anyone can read them.
Based on this jQuery autocomplete tutorial
Going further: how to implement this with multiple lines textarea?
Yesterday I received a mail (thanks, Robert!) asking how to modify code to fit another requirement: multiple advisors, same textarea field. This can be easily achieved with small modifications in lookup and fill functions.
This is the new code:
function lookup(inputString) { var direct = $("textarea[name=DEMOTHE_SUPERV]").val(); var cuantos = 0; if(direct.length == 0) { $('#suggestions').hide(); } else { cuantos = direct.split("\n").length; if (cuantos>1){ ultima = direct.split("\n"); ultima = ultima[cuantos - 1]; } else{ ultima = direct; } // for multiple lines in the textarea, use parameter ultima in $.post // for single lines in the textarea, use parameter direct in $.post $.post("/preview/autocompletar.php", {queryString: ""+ultima+""}, function(data){ if(data.length >0) { $('#suggestions').show(); $('#autoSuggestionsList').html(data); } }); } } // lookup function fill(thisValue) { if (!thisValue) return 0; var actual = $("textarea[name=DEMOTHE_SUPERV]").val(); cuantos = actual.split("\n").length; valores = actual.split("\n"); var elvalor = ""; if (cuantos > 1){ // ya hay valores, respetar todos y sustituir el ultimo por lo clickado for ($i=0; $i <h2>Files</h2> <strong>In your HTML</strong> Include the path to the autocomplete.js: <pre lang="javascript"><script lang="text/javascript" src="/preview/preview.js"><!--mce:2--></script> |
THE JAVASCRIPT (autocomplete.js)
function initializeAutoComplete(){ var eldirectores = $("textarea[name=DEMOTHE_SUPERV]"); if ( eldirectores ){ // si el textarea DEMOTHE_SUPERV forma parte del formulario... //eldirectores.attr("onkeyup",lookup()); eldirectores.keyup(function(){ lookup(); }); eldirectores.click(function(){ fill(); }); } } function lookup(inputString) { var direct = $("textarea[name=DEMOTHE_SUPERV]").val(); var cuantos = 0; if(direct.length == 0) { // Si no se ha escrito nada en la casilla, esconder la zona de sugerencias. $('#suggestions').hide(); } else { //alert('onkeyup: val='+direct); cuantos = direct.split("\n").length; if (cuantos>1){ ultima = direct.split("\n"); ultima = ultima[cuantos - 1]; } else{ ultima = direct; } // for multiple lines in the textarea, use parameter ultima in $.post // for single lines in the textarea, use parameter direct in $.post // ALTER HERE!!! Change your path to autocomplete php script!! $.post("/preview/autocompletar.php", {queryString: ""+ultima+""}, function(data){ if(data.length >0) { $('#suggestions').show(); $('#autoSuggestionsList').html(data); } }); // pongo esto para que se quede desactivada hoy // return 0; } } // lookup function fill(thisValue) { if (!thisValue) return 0; var actual = $("textarea[name=DEMOTHE_SUPERV]").val(); cuantos = actual.split("\n").length; valores = actual.split("\n"); var elvalor = ""; if (cuantos > 1){ // ya hay valores, respetar todos y sustituir el ultimo por lo clickado for ($i=0; $i <strong>THE PHP (autocompletar.php)</strong> <pre lang="php"><!--?php $db = new mysqli('localhost', 'yourUsername' ,'yourPassword', 'cdsinvenio'); if(!$db) { echo 'ERROR: No se ha podido conectar a la BD.'; } else { // Is there a posted query string? if(isset($_POST['queryString'])) { $queryString = $db--->real_escape_string($_POST['queryString']); // Is the string length greater than 0? if(strlen($queryString) >0) { $query = $db->query("SET NAMES 'utf8'"); // YOU NEED TO ALTER THE QUERY TO MATCH YOUR DATABASE. // eg: SELECT yourColumnName FROM yourTable WHERE yourColumnName LIKE '$queryString%' LIMIT 10 $query = "SELECT DISTINCT BINARY value,value FROM cdsinvenio.bib70x,cdsinvenio.bibrec_bib70x WHERE bib70x.value LIKE '$queryString%' and tag='700__a' and bib70x.id = bibrec_bib70x.id_bibxxx AND bibrec_bib70x.id_bibrec NOT IN (SELECT DISTINCT bibrec_bib98x.id_bibrec from bibrec_bib98x WHERE id_bibxxx=3) LIMIT 15"; $query = $db->query($query); if($query) { // While there are results loop through them - fetching an Object. while ($result = $query ->fetch_object()) { // Format the results, im using <li> for the list, you can change it. // The onClick function fills the textbox with the result. echo '</li> <li onclick="fill(\''.$result->value.'\');">'.$result->value.'</li> '; } } else { echo 'ERROR: Problemas al obtener resultados.'; } } else { // Dont do anything. } // There is a queryString. } else { echo 'Do not call this script directly'; } } ?> |
FINAL CODE
Updated 2011-06-06: Changed the implementation to autocomplete several fields. Final implementation is below
autocomplete.js
/* * Will initialize the form textarea fields that will suggest values to autocomplete * ---- Sets the behaviour for keyup and click * ---- Appends the suggestionbox to every textarea field */ function initializeAutoComplete(){ var eldirectores = $("textarea[name=DEMOTHE_SUPERV]"); if ( eldirectores ){ eldirectores.keyup(function(){ lookup2('DEMOTHE_SUPERV','700__a'); }); eldirectores.click(function(){ fill2('DEMOTHE_SUPERV'); }); eldirectores.after(' <div id="suggestions_DEMOTHE_SUPERV" class="suggestionsBox" style="display: none;"><img style="position: relative; top: -12 px; left: 30px;" src="/preview/upArrow.png" alt="upArrow" /> <div id="autoSuggestionsList_DEMOTHE_SUPERV" class="suggestionList"></div> </div> '); } var elponentes = $("textarea[name=PONENTE]"); if ( elponentes ){ elponentes.keyup(function(){ lookup2('PONENTE','7202_a'); }); elponentes.click(function(){ fill2('PONENTE'); }); elponentes.after(' <div id="suggestions_PONENTE" class="suggestionsBox" style="display: none;"><img style="position: relative; top: -12px; left: 30px;" src="/preview/upArrow.png" alt="upArrow" /> <div id="autoSuggestionsList_PONENTE" class="suggestionList"></div> </div> '); } var laskeywords = $("textarea[name=DEMOTHE_KEYWRDS]"); if ( laskeywords ){ laskeywords.keyup(function(){ lookup2('DEMOTHE_KEYWRDS','6531_a'); }); laskeywords.click(function(){ fill2('DEMOTHE_KEYWRDS'); }); laskeywords.after(' <div id="suggestions_DEMOTHE_KEYWRDS" class="suggestionsBox" style="display: none;"><img style="position: relative; top: -12 px; left: 30px;" src="/preview/upArrow.png" alt="upArrow" /> <div id="autoSuggestionsList_DEMOTHE_KEYWRDS" class="suggestionList"></div> </div> '); } } function lookup2(nombre,etiqueta) { var direct = $("textarea[name="+nombre+"]").val(); var cuantos = 0; if(direct.length == 0) { $('#suggestions_'+nombre).hide(); } else { cuantos = direct.split("\n").length; if (cuantos>1){ ultima = direct.split("\n"); ultima = ultima[cuantos - 1]; } else{ ultima = direct; } $.post("/preview/autocompletar2.php", {queryString: ""+ultima+"", tag: ""+etiqueta+"", midiv: ""+nombre+""}, function(data){ if(data.length >0) { $('#suggestions_'+nombre).show(); $('#autoSuggestionsList_'+nombre).html(data); } }); } } // lookup2 function fill2(nombre,thisValue) { if (!thisValue) return 0; if (!nombre) return -1; var actual = $("textarea[name="+nombre+"]").val(); cuantos = actual.split("\n").length; valores = actual.split("\n"); var elvalor = ""; if (cuantos > 1){ // ya hay valores, respetar todos y sustituir el ultimo por lo clickado for ($i=0; $i<cuantos-1; $i++){ elvalor = elvalor + valores[$i] + "\n"; } elvalor = elvalor + thisValue; } else{ elvalor = thisValue; } $("textarea[name="+nombre+"]").val(elvalor); vartmp = "#suggestions_"+nombre; setTimeout("$(vartmp).hide();", 200); } |
autocomplete.php
<!--?php // ALTER HERE!!! // $db = new mysqli('localhost', 'yourUsername', 'yourPassword', 'yourDatabase'); if(!$db) { echo 'ERROR: No se ha podido conectar a la BD.'; } else { if (!isset($_POST['tag'])){ die("ERROR: No se ha enviado el parametro tag"); } // Is there a posted query string? if(isset($_POST['queryString'])) { $queryString = $db--->real_escape_string($_POST['queryString']); $tag= $db->real_escape_string($_POST['tag']); $midiv=$_POST['midiv']; if ( !is_numeric(substr($tag,0,2)) ) return -1; $bibtablename="cdsinvenio.bib".substr($tag,0,2)."x"; $bibrectablename="cdsinvenio.bibrec_bib".substr($tag,0,2)."x"; if(strlen($queryString) >0) { $query = $db->query("SET NAMES 'utf8'"); $query = "SELECT DISTINCT BINARY value,value FROM ".$bibtablename.",".$bibrectablename." WHERE ".$bibtablename.".value LIKE '$queryString% ' and tag='$tag' and ".$bibtablename.".id = ".$bibrectablename.".id_bibxxx AND ".$bibrectablename.".id_bibrec NOT IN (SELECT DISTINCT bibrec_bib98x.id_bibrec from bibrec_ bib98x WHERE id_bibxxx=3) LIMIT 15"; $query = $db->query($query); if($query) { // While there are results loop through them - fetching an Object. while ($result = $query ->fetch_object()) { echo '<li onclick="fill2(\''.$midiv.'\',\''.$result->value.'\');">'.$result->value.'</li>'; } } else { echo 'ERROR: Problemas al obtener resultados.'; } } else { // Dont do anything. } // There is a queryString. } else { echo 'No should not call this script directly!'; } } |
Styles.css (add to your invenio stylesheet)
.suggestionsBox { position: relative; left: 30px; margin: 10px 0px 0px 0px; width: 200px; background-color: #212427; -moz-border-radius: 7px; -webkit-border-radius: 7px; border: 2px solid #000; color: #fff; font-size: 0.8em; } .suggestionList { margin: 0px; padding: 0px; } .suggestionList li { margin: 0px 0px 3px 0px; padding: 3px; cursor: pointer; } .suggestionList li:hover { background-color: #659CD8; } |
New invenio element description Called AUTOC, element type=response
text=''' <script type="text/javascript"> $(document).ready(function(){ a = initializeAutoComplete(); }); ''' </script> |
autocomplete.php v2
Another version of autocomplete.php with more functionalities (able to search for values in more tables):
<?php $db = new mysqli('localhost', 'yourUsername' ,'yourPassword', 'yourDatabase'); if(!$db) { echo 'ERROR: No se ha podido conectar a la BD.'; } else { if (!isset($_POST['tag'])){ die("ERROR: No se ha enviado el parametro tag"); } // Is there a posted query string? if(isset($_POST['queryString'])) { $queryString = $db->real_escape_string($_POST['queryString']); $tag= $db->real_escape_string($_POST['tag']); $midiv=$_POST['midiv']; // p ejemplo para tag = 700__a $bibtablename=cdsinvenio.bib70x y $bibrectablename=cdsinvenio.bibrec_bib70x if ( !is_numeric(substr($tag,0,2)) ) return -1; $bibtablename="cdsinvenio.bib".substr($tag,0,2)."x"; $bibrectablename="cdsinvenio.bibrec_bib".substr($tag,0,2)."x"; // Is the string length greater than 0? if(strlen($queryString) >0) { $query = $db->query("SET NAMES 'utf8'"); // Sacar autores de todas las etiquetas? (700,720,...)? // si la etiqueta que busco no es de palabra clave, es que busco artículos... asi que dale caña a todos los autores $alltheauthors = ( ($_POST['tag'] != '6531_a') ); if ($alltheauthors){ $query = "(SELECT DISTINCT BINARY value, value FROM cdsinvenio.uz_PDI WHERE uz_PDI.value LIKE '$queryString%') UNION ALL (SELECT DISTINCT BINARY value, value FROM cdsinvenio.bib70x, cdsinvenio.bibrec_bib70x WHERE bib70x.value LIKE '$queryString%' AND tag='700__a' AND bib70x.id = bibrec_bib70x.id_bibxxx AND bibrec_bib70x.id_bibrec NOT IN (SELECT DISTINCT bibrec_bib98x.id_bibrec from bibrec_bib98x WHERE id_bibxxx=3)) UNION ALL (SELECT DISTINCT BINARY value, value FROM cdsinvenio.bib72x, cdsinvenio.bibrec_bib72x WHERE bib72x.value LIKE '$queryString%' AND tag='7202_a' AND bib72x.id = bibrec_bib72x.id_bibxxx AND bibrec_bib72x.id_bibrec NOT IN (SELECT DISTINCT bibrec_bib98x.id_bibrec from bibrec_bib98x WHERE id_bibxxx=3)) ORDER BY VALUE LIMIT 15;"; } else{ $query = "SELECT DISTINCT BINARY value,value FROM ".$bibtablename.",".$bibrectablename." WHERE ".$bibtablename.".value LIKE '$queryString%' and tag='$tag' and ".$bibtablename.".id = ".$bibrectablename.".id_bibxxx AND ".$bibrectablename.".id_bibrec NOT IN (SELECT DISTINCT bibrec_bib98x.id_bibrec from bibrec_bib98x WHERE id_bibxxx=3) LIMIT 15"; } $query = $db->query($query); if($query) { // While there are results loop through them - fetching an Object. while ($result = $query ->fetch_object()) { echo '<li onClick="fill2(\''.$midiv.'\',\''.$result->value.'\');">'.$result->value.'</li>'; } } else { echo 'ERROR: Problemas al obtener resultados.'; } } else { // Dont do anything. } // There is a queryString. } else { echo 'No deberias estar viendo este texto!'; } } ?> |