Questions for Dr. Rubec about the IMA CDT and resulting data

[Anonymous]

Peter,

At risk of being called an employee of MAC, or a mindless MAC-backer I will venture to ask a few questions regarding the IMA ISE based CDT and some of the data resulting from it.

1) What is the lower reliable method detection limit of the ISE method?

2) What error ranges are ascribed to values obtained by the ISE method?

3) In a personal correspondence made by C. Ellen Gonter, that you provided to me, there was a comment: “Statistics from these [round robin] studies indicate there is no difference between 0.03 and 5.0 mg CN/L using the ISE procedure”. Is she saying that values between 0.03 and 5.0 mg/L all have similar statistical value or that the method cannot statistically differentiate between known values of 0.03 mg/L and 5.0 mg/L?

4) If the ISE method cannot differentiate between values of 0.03 mg/L and 5.0 mg/L why is the BFAR legal enforcement value set at 0.2 mg/L?

5) In Rubec (2002) there is discussion of cyanide absence and presence in fish by Family (Tables 3A & 3B). In the table fish are listed as “CN absent” or “CN present”. What was the cut-off value for the determination of absence of presence of cyanide?

6) In Rubec (2001) there is a discussion of cyanide testing on marine-aquarium fish that lumps fish into 3 categories (Table 2). The categories are: “Positive, >=0.2 mg/Kg”, “Trace, >0 to <0.2 mg/Kg”, and “Zero, 0 mg/Kg”. Shouldn’t “Trace, >0 to <0.2 mg/Kg”, actually read “Trace, above lower method detection limit to <0.2 mg/Kg”? Also shouldn’t “Zero, 0 mg/Kg” actually be specified as “below method detection limit”?

Thank you for your participation in this board and for considering my questions.

Respectfully,
Lee Morey


Rubec, P.J., F. Cruz, V. Pratt, M. Oellers, B. McCullough, and F. Lallo. 2001. Cyanide-free net-caught fish for the marine aquarium trade. Aquarium Sciences and Conservation 3: 37-51.

Rubec, P.J., V. Pratt, B. McCullough, B. Manipula, J. Alban, T. Espero, and E. Suplido. 2002. Trends Determined By Cyanide Testing On Marine Aquarium Fish In the Philippines. Draft.

 

[PeterIMA]

Answers to questions posed by Lee Morey (SciGuy)
1) What is the lower reliable method detection limit of the ISE method?

Rubec-The determination of total cyanide ion concentrations tied to use of an Ion Selective Electrode (ISE) was extensively evaluated by the American Society of Testing and Materiasl (ASTM) and the American Public Health Association (APHA). It was found that with a four point calibration curve that the relationship on a semi-log plot was linear (straight line) over the range 0.03 mg/L to 10 mg/L. Hence, one can consider the lower limit to reliably determine the cyanide ion concentration in solution to be 0.03 mg/L.

Evaluations done by the IMA found similar linear relationships over this range with daily calibrations done in the six CDT laboratories (e.g, 0.01 mg/L was found to depart from the straight line calibration curve). Below 0.03 mg/L cyanide can be detected. However, it becomes difficult to determine what concentration that represents (because the concentration cannot be easily determined from the straight line calibration curve on a semi-log plot)

2) What error ranges are ascribed to values obtained by the ISE method?

Rubec- I am not familiar with this being done by any analytical laboratory that has used this method. I will check into the documentation provided in Standard Methods For the Examination Of Water and Wastewater 20th Edition published by the American Public Health Association (APHA 1998). To assign error ranges one would need to do repeated tests on the same sample. Error ranges are generally determined on a minimum of 30 samples (in a statistical sense). This is impractical. Some replicate testing was done by IMA and samples were exchanged between the laboratories. Very similar results were found.

3) In a personal correspondence made by C. Ellen Gonter, that you provided to me, there was a comment: &#8220;Statistics from these [round robin] studies indicate there is no difference between 0.03 and 5.0 mg CN/L using the ISE procedure&#8221;. Is she saying that values between 0.03 and 5.0 mg/L all have similar statistical value or that the method cannot statistically differentiate between known values of 0.03 mg/L and 5.0 mg/L?

Rubec-This pertains to a letter sent to me by Dr. Gonter (who was on the MAC scientific review panel). She is the foremost expert on the ISE test, since she helped to develop the method. I agree that the statement is ambiguous. I know from my telephone conversations with her that she does not consider that there is "no difference" between the two values listed above. There definitely is a marked difference between 0.3 mg/L and 5.0 mg/L, and the Total Cyanide Ion method tied to ISE can definitely differentiate the difference. I don't believe her comments pertain to statistics. Most chemists don't involve themselves with statistics. Rather, they are concerned with reliability of the method, lower levels of detection, replicability of the method, QA/QC etc.

Dr. Gonter found no problem with the original SOP created (from the APHA book Standard Methods) and used by the IMA, other than the fact that the some of the sentences in the manual were poor English (since they were written by a Filipino chemist). The actual method works and the results obtained by the IMA were considered by Dr. Gonter to be reliable. They also were considered reliable by Dr. Martin Frant (who developed the ISE electrode and meter for Orion, and also was on the MAC panel). I also provided you with a letter from Dr. George Dixon of the University of Waterloo that supported the SOP used by IMA and was critical of the MAC report created by Paul Holthus (the latter is seriously biased). I find it odd that you don't refer to this information.

4) If the ISE method cannot differentiate between values of 0.03 mg/L and 5.0 mg/L why is the BFAR legal enforcement value set at 0.2 mg/L?

The method can easily differentiate between 0.03 mg/L and 5.0 mg/L (as I stated above) using the calibration discussed above. It has not much to do with the BFAR using 0.2 mg/kg for enforcement purposes. The test can differentiate levels lower than 0.2 mg/L from the calibration curve. My previous statements on Reefs.org were there is no scientific basis for setting the level for enforcement at 0.2 mg/kg (note this is not the same as 0.2 mg/L).

A calculation needs to be done after the cyanide ion concentration in solution is determined using the ISE probe linked to an ISE meter. This calculation determines the concentration in relation to the wet weight of tissue (based on the known weight of the tissue sample digested and analyzed) and the volume containing the cyanide ion in sodium hydroxide solution (the concentration in solution). It is necessary to calculate the concentration on a weight basis from the concentration in solution.

5) In Rubec (2002) there is discussion of cyanide absence and presence in fish by Family (Tables 3A & 3B). In the table fish are listed as &#8220;CN absent&#8221; or &#8220;CN present&#8221;. What was the cut-off value for the determination of absence of presence of cyanide?

There was no cut-off used in the CDT paper published in Collection, Culture and Conservation (Rubec et al. 2003). Absence means Zero (0 mg/kg) and Presence was defined as everything above Zero (> 0 mg/kg). Cyanide was detected and scored "Present" or it was not detected and was scored Absent.

6) In Rubec (2001) there is a discussion of cyanide testing on marine-aquarium fish that lumps fish into 3 categories (Table 2). The categories are: &#8220;Positive, >=0.2 mg/Kg&#8221;, &#8220;Trace, >0 to <0.2 mg/Kg&#8221;, and &#8220;Zero, 0 mg/Kg&#8221;. Shouldn&#8217;t &#8220;Trace, >0 to <0.2 mg/Kg&#8221;, actually read &#8220;Trace, above lower method detection limit to <0.2 mg/Kg&#8221;? Also shouldn&#8217;t &#8220;Zero, 0 mg/Kg&#8221; actually be specified as &#8220;below method detection limit&#8221;?

At the time (2000) that I wrote the paper (Cyanide-Free Net-Caught published in 2001) the IMA was still doing CDT work. The method described above reflects testing being done by IMA. In the four Standard Operations Manuals (SOPs) that I helped to prepare (issued in September 2001) the recommended scoring is as follows. There are are 4 classes: A-Positive (>=0.2 mg/kg), B-Trace (>=0.03 mg/kg and <0.2 mg/kg), Below Detectable Limits (BDL) (>0 and <0.03 mg/kg), and D-Zero (0 mg/kg).

I assume that is how BFAR is presently scoring the data (provided they are following the SOP manuals provided to them by IMA). I should note that the IMA recorded all of the raw data [including the weight of the sample, the ISE reading (electrical potential converted to mg/L), the calculated cyanide concentration in mg/kg, and the derived scores into a MicroSoft Access database]. Presumably, BFAR is doing the same, since they apparently are utilizing the same ISE methodology previously used by IMA.


Rubec, P.J., F. Cruz, V. Pratt, M. Oellers, B. McCullough, and F. Lallo. 2001. Cyanide-free net-caught fish for the marine aquarium trade. Aquarium Sciences and Conservation 3: 37-51.

Rubec, P.J., V. Pratt, B. McCullough, B. Manipula, J. Alban, T. Espero, and E. Suplido. 2003. Pages, 327-340, In: J.C. Cato and C.L. Brown (eds)
Trends Determined By Cyanide Testing On Marine Aquarium Fish In the Philippines, Iowa State Press, Ames, Iowa.

Sincerely,
Peter J. Rubec, Ph.D.

 

[PeterIMA]

Further comment about Dr. Gonter's letter. She was stating that the different laboratories that did the round-robin comparisons of tissue samples (sent to all the laboratories) found no statistical differences in their analytical results using the Total Cyanide Ion method with ISE apparatus. The different labs found very similar test results over the range 0.03 to 10 mg/L. This is the range over which the calibration curve was found to be linear (as I previously mentioned). Dr. Gonter was mistaken to list the range as being 0.03 to 5.0 mg/L in her letter to me. (Apparently I am wrong and Gonter had it right in her letter, since 0.03 to 5.0 mg/L was the range for the round-robin comparisons as discussed in my subsequent posting below).

Peter Rubec

 

[PeterIMA]

Lee Morey asked "What error ranges are associated with the ISE method?"

I stated that I would consult Standard Methods For the Examination of Water and Wastewater, which has the information concerning the round-robin evaluation conducted on the CN ion-selective electrode (ISE) method using the absorption solution from total cyanide distillation (ASTM 1987). The information is presented in the 18th edition (APHA 1992) and the 20th Edition (APHA 1998) under the headings "Ion Selective Electrode/Precision and Bias"

The round robin evaluation involved different laboratories analyzing unknown cyanide concentrations provided to them from the same source (original samples subdivided). Based on the results of six operators in five laboratories, the overall and single-operator precision of this method within its designated ranges may be expressed as follows:

Reagent water:
Sl = 0.06x + 0.003
So = 0.03x + 0.008

Selected water matrices:
Sl = 0.05x + 0.008
So = 0.03x + 0.012

Where:
Sl = overall precision, mg/L ,
So = single operator precision, mg/L, and
x = cyanide concentration, mg/L.

Recoveries of cyanide from reagent water, and selected water matrices (coke plant and refinery wastes, sewage and surface water) were presented in a table.

The concentrations of cyanide ion added to reagent water in the distillation flasks in the five laboratories were respectively 0.060, 0.500, 0.900, and 5.00 mg/L. The mean recoveries of cyanide ion determined were respectively 0.059, 0.459, 0.911, and 5.07 mg/L based on 18 samples for each concentration tested. Bias for the four concentrations were respectively -0.001, -0.041, 0.011, and 0.07 mg/L. The % bias were respectively 2, -8, 1, and 1.

The concentrations of cyanide ion added to selected water matrices in the distillations flasks were the same as above (0.060, 0.500, 0.900, and 5.00 mg/L). Recovery concentrations were respectively 0.058, 0.468, 0.922, and 5.13 mg/L based on 14, 21, 19, and 20 respective samples tested associated with the concentrations previously listed. Bias for the four concentrations were respectively -0.002, -0.032, 0.022, and 0.13 mg/L. The % bias were respectively -3, -6, 2, and 3.

APHA 1992. Standard Methods for the Examination of Water and Waster 18th Edition. 4500-CN- Cyanide, Prepared by the American Public Health Association, American Water Works Association, and Water Environmental Federation, Editorial Board A.E. Green, L.S. Clescari, A.E. Eaton. Part 4, Page 4-27.

APHA 1998. Standard Methods for the Examination of Water and Waster 20th Edition. 4500-CN- Cyanide. Prepared by the American Public Health Association, American Water Works Association, and Water Environmental Federation. Editorial Board A.E. Green, L.S. Clescari, A.E. Eaton, Part 4, Page 4-41.

ASTM (American Association of Testing And Materials) 1987. Research Report D0236:19-1131. American Society of Testing and Materials, Philadelphia, Pa.

Sincerely,
Peter Rubec, Ph.D.

 

[Anonymous]

Peter,

Thank you for your in depth and thoughtful answer. I've been thinking about what you have presented for the last week.

-Based on your presentation how would you present a value including the uncertainty?

-Would you use the extreme bias numbers (+3% and -8%) as the uncertainty?

-Should the standard deviation of the mean be incorporated in the calculations of the bias? Aren't we talking about an "average of averages" when looking at the bias numbers you presented?

-If it is possible that the test method can give values higher than the actual value shouldn't this be considered by BFAR in legal arguements? For instance, if the test method can potentially overestimate the value by 3% shouldn't BFAR only consider prosecution only if analytical values are greater than (0.2 mg/kg) + 0.03 x (0.2 mg/kg) = 0.206 mg/kg? Perhaps the BFAR legal action limit of 0.2 mg/kg been set at its present (and arguably high) value to exclude any possible uncertainty in the analytical process?

Thanks again,
Lee Morey

 

[PeterIMA]

Response to Lee Morey,

-Based on your presentation how would you present a value including the uncertainty?
Answer-Based on the % bias data for the various test samples summarized from the ASTM round robin study, I think one could state that the %bias was about +/-3%.

-Would you use the extreme bias numbers (+3% and -8%) as the uncertainty?
Answer-No, I don't think the extreme bias numbers are the actual uncertainty. As a biologist, I generally express uncertainly within 95% confidence intervals or within 2 standard deviations from the mean. The chemists at the laboratories associated with the round-robin studies express bias differently (as I presented in my previous posting).

Would the standard deviation of the mean be incorporated in the calculations of the bias? Aren't we talking about an "average of averages" when looking at the bias numbers you presented?
Answer-As far as I can tell from looking at the table I summarized from APHA, they calculated how the mean value of the replicate measurements departed from the known concentration subjected to testing. I don't see that as an "average of averages". This was expressed as the bias and then as the % bias. For example the unknown concentration tested was let say 4.2 mg/L. Twenty-five replicate samples were tested and the average was 4.0 mg/L. The difference between the original concentration and the mean estimated concentration was 0.2 mg/L (the bias). This can be expressed as a % bias by calculating 0.2 X100/4.2=4.76%. Since the mean estimate was lower than the actual concentration, this can be expressed with a minus sign as -4.76%.

-If it is possible that the test method can give values higher than the actual value shouldn't this be considered by BFAR in legal arguments? For instance, if the test method can potentially overestimate the value by 3% shouldn't BFAR only consider prosecution only if analytical values are greater than (0.2 mg/kg) + 0.03 x (0.2 mg/kg) = 0.206 mg/kg? Perhaps the BFAR legal action limit of 0.2 mg/kg been set at its present (and arguably high) value to exclude any possible uncertainty in the analytical process?
Answer- I can't speak for either BFAR or the Philippine court system. I would agree that one could argue that any measurement has some bias due to the precision of the analytical method. Hence a value of 0.2 mg/kg could be interpreted to lie somewhere between 0.194 and 0.206 mg/kg. These numbers were derived by taking 3% of 0.2 mg/kg (as you apparently did). However, I never had any conversations
with BFAR staff or saw any documentation to indicate to me that this was ever considered by either BFAR or the Philippine court system. I don't believe it has any bearing on why they chose 0.2 mg/kg and above as the basis for prosecution purposes. My point made previously was that there does not appear to be any scientific basis for the 0.2 mg/kg used to support prosecution of cyanide fishermen. Concentrations lower than this also indicate that the fish were exposed to cyanide (most probably from cyanide fishing). However, I would agree that there should be some cut-off concentration, below which the fisherman is not prosecuted. In my opinion, the cut-off concentration (0.2mg/kg) adopted is too high.

Peter Rubec

 

[Anonymous]

Peter,

After seeing you present several formulas for precision and a detailed discussion of bias I became confused. I went back to my bookcase. Bias doesn’t seem to directly address the uncertainty of the method.

I have the 1975 version of the Standard Methods For the Examination of Water and Wastewater (ALPHA, 1975). Previously I thought that it didn't include information about the ISE method. On closer review I found that it does. It has 5 pages on the ISE-CDT. You are quoting from a more recent version than mine but now I believe that I understand the generals of calculating the precision for the test. Here’s what ALPHA, 1975 has to say about the precision of the test.

Precision

The precision of the cyanide ion selective electrode method was determined from four levels of cyanide concentration by five laboratories and seven analysts. The precision of the method within its designated range [0.05 to 10 mg/l CN] may be expressed as follows:

ST=0.113X+0.024

where ST=overall precision and X=CN concentration, mg/l.

The more current ALPHA, 1998 that you reference uses the variable SI instead of ST to refer to overall precision and provides the following equation to calculate precision for selected water matrices.

SI = 0.05X + 0.008

Precision is the same thing as relative uncertainty (Taylor, 1982). So, calculating the uncertainty is simple. If the meter in the laboratory was found to read 0.20 mg/l then the uncertainty using ALPHA, 1998 for selected water matrices would be:

ST = (0.05)(0.20 mg/l)+0.008 = 0.018 mg/l;

The fractional uncertainty would be (0.018 mg/l)/(0.20 mg/l) = .09

The percentage uncertainty would be fractional uncertainty multiplied by 100% (Taylor, 1982). In this case:

(100%)(0.018 mg/l)/(0.20 mg/l) = 9%

Thus, if a sample tested as having 0.20 mg/l CN one would say that the sample is:

0.20 mg/l +/- 0.018 mg/l

-or-

0.2 mg/l +/- 9%

Sincerely,
-Lee

APHA 1975. Standard Methods for the Examination of Water and Waster 14th Edition. Prepared by the American Public Health Association, American Water Works Association, and Water Pollution Control Federation. Editorial Board M.C. Rand, A.E. Greenberg, M.J. Teras, Page 376.

APHA 1998. Standard Methods for the Examination of Water and Waster 20th Edition. 4500-CN- Cyanide. Prepared by the American Public Health Association, American Water Works Association, and Water Environmental Federation. Editorial Board A.E. Green, L.S. Clescari, A.E. Eaton, Part 4, Page 4-41.

Taylor, John R. 1982. An Introduction to Error Analysis: The Study of Uncertainties in Physical Measurements. University Science Books. Page 29.

 

[Anonymous]

Here is a table of uncertainties based on ALPHA, 1998 that I put together.

Value........Uncertainty................Percent Uncertainty
(mg/l).......(mg/l)..........................%

0.03..........0.0095......................32%
0.05..........0.0105......................21%
0.10..........0.013........................13%
0.20..........0.018..........................9%
0.30..........0.023..........................8%
0.40..........0.028..........................7%
0.50..........0.033..........................7%
0.60..........0.038..........................6%
0.70..........0.043..........................6%
0.80..........0.048..........................6%
0.90..........0.053..........................6%
1.00..........0.058..........................6%
5.00..........0.258..........................5%
10.00........0.508..........................5%

Thus it is accurate to say that the percent uncertainty of the test over the range of 0.03 mg/l to 10 mg/l was found to be from 5% to 32%.


Sincerely,
Lee

 

[PeterIMA]

Lee, Thanks for the clarification. I would be interested in obtaining a photocopy of the information from the APHA (1975) that you are quoting.
Your previous questions clouded the way uncertainty was being expressed by talking about confidence limits and standard error. I tried to indicate that was not the way the chemists had expressed it (by discussing the % bias).I am somewhat surprised that there would be a discussion about precision and uncertainty of the ISE methodology in the 1975 edition of Standard Methods, since I do not recall seeing it in the 1985 edition. As far as I can recall, the ISE electrodes for cyanide ion were only developed by Dr. Martin Frant of Orion in the early 1970s. So, it is surprising to me that it would be in the 1975 edition.

I appreciate your clarification about uncertainty. I accept that the uncertainty increases at lower concentrations of cyanide ion being detected. What do you think that BFAR should do about this?

Peter Rubec

 

[Anonymous]

Peter,

I'd be happy to fax or mail you a copy of the appropriate pages from ALPHA, 1975. Please P.M. the appropriate fax number or mail address.

Please note that my table is based on ALPHA, 1998.

It is interesting to note that the ALPHA, 1975 process specifies in the apparatus section the use of an Orion Model 94-06A ISE or equivalent.

I'll have to think a bit about what I'd hypothetically say to BFAR. I know that I'd say that the maximum uncertainty should be considered.

-Lee

 

[Anonymous]

Peter,

A person would need to know the density of the sample to really address your question. If the fish flesh density was the same as pure water (density = 1) then mg/l would equal mg/kg and we could use simple algebra to determine a hypothetical answer to BFAR.

What is the density, specific gravity or other conversion factor that the BFAR labs use to convert from mg/l to mg/kg when running the CDT for ornamental fish? Presumably the ISE gives an answer in mg/l which is not directly comparable to P.I. legal action limit of 0.20 mg/kg.

Sincerely,
-Lee

 

[PeterIMA]

Lee, Just a couple of points. First, shouldn't you be stating APHA as the acronym for the American Public Health Association instead of ALPHA?

I already explained that the wet weight of tissue digested needs to be used in the calculation along with the volumetric concentration in mg/l to determine the cyanide ion concentration in mg/kg.

While your discussion of precision (uncertainty) is of interest, it is not as relevant as whether cyanide was detected (present) or was not present (absent). In my mind, there has been too much focus on the cyanide ion concentration (and precision of these measurements) rather that the fact that cyanide was being detected. Any concentration of cyanide detected was (is) indicative that the fish was exposed to cyanide. Since, cyanide in its unbound form does not occur naturally in the marine environment, the test results are a stong indication that cyanide was being used by the fishermen. Generally, the test results were used along with other evidence, such as the fishermen being caught with cyanide or cyanide squirt bottles in their possession. Lets not lose sight of this, as we discuss these issues.

Peter Rubec

 

[PeterIMA]

edited double posting

 

[Anonymous]

Lee, Just a couple of points. First, shouldn't you be stating APHA as the acronym for the American Public Health Association instead of ALPHA?

Yes, my mistake. I should have been using APHA, not ALPHA.

I already explained that the wet weight of tissue digested needs to be used in the calculation along with the volumetric concentration in mg/l to determine the cyanide ion concentration in mg/kg.

Understood and acknowledged.

While your discussion of precision (uncertainty) is of interest, it is not as relevant as whether cyanide was detected (present) or was not present (absent). In my mind, there has been too much focus on the cyanide ion concentration (and precision of these measurements) rather that the fact that cyanide was being detected. Any concentration of cyanide detected was (is) indicative that the fish was exposed to cyanide. Since, cyanide in its unbound form does not occur naturally in the marine environment, the test results are a stong indication that cyanide was being used by the fishermen. Generally, the test results were used along with other evidence, such as the fishermen being caught with cyanide or cyanide squirt bottles in their possession. Lets not lose sight of this, as we discuss these issues.

Peter Rubec

Agreed, I'll do my best to not lose sight of your point. However, please understand my point that analytical data, or any numerical data, only has any true value if it's limitations are fully realized and that the data is looked at in the proper environmental context.

Sincerely,
-Lee