sql - Why does Sybase ASE throw a truncation error in a division followed by a subtraction? -

-

i using sybase ase 15. in following sql, there truncation error, if division of 2 numbers followed subtraction.

    declare @p1  decimal (30,16)     declare @p2  decimal (30,16)     select @p1 = 0.0000504412630951  --sixteen decimal places     select @p2 = 0.0000512178647912     select 'this succeeds:'     select @p1, @p2, (@p1 / @p2)          --this succeeds: displays values     select 'this fails:'     select @p1, @p2, (@p1 / @p2)  - 1.0  --this fails truncation error.  why?     select 'why succeed?'     select @p1, @p2, round( (@p1 / @p2), 46 )  - 1.0  --this succeeds 46 or less, fails 47 or greater - why?

thanks in advance.

bob

the 1.0 not decimal datatype @p1 , @p2 , it's created float since not specify precision , scale (or cast numeric/decimal data type). first attempt succeeds because there no implicit conversion, you're dividing 2 decimals same precision/scale. 2nd attempt fails because subtracting 1.0 subtracting float value of 1.0 against decimal forces implicit conversion decimal. since implied , not explicit, truncation error because system wants know losing scale doing given operation. round function in 3rd attempt doing explicit conversion, truncation ignored because telling system want out of operation.

you can find more information on scale error @ link below, how turn off if don't care (look towards bottom of page).

sybooks online - conversion error handling


Comments

Post a Comment

Popular posts from this blog

php - render data via PDO::FETCH_FUNC vs loop -

-
i've been using 2 methods render data. the first one: function name($id,$name){ return '<div id="'.$id.'">'.$name.'</div>'; } echo implode($pdo->query("select id,name user")->fetchall(pdo::fetch_func,'name')); the second one: $users = $pdo->query("select id,name user")->fetchall(pdo::fetch_obj); foreach($users $user){ echo name($user->id,$user->name); } i don't understand how pdo::fetch_func works. tried figure out. however, not well-documented. could please explain fetch mode? , also, 1 performs better? thank you. both methods wrong , have learn how use templates , how separate business logic presentation logic. $users = $pdo->query("select id,name user")->fetchall(pdo::fetch_obj); tpl::assign('users', $users); is code business logic part. then in template <?php foreach $users $row): ?> <div id="<?=$...
Read more

c++ - OpenCV Error: Assertion failed <scn == 3 ::scn == 4> in unknown function, -

-
i read alot other solution still confused should mine... #include <opencv2/objdetect/objdetect.hpp> #include <opencv2/highgui/highgui.hpp> #include <opencv2/imgproc/imgproc.hpp> #include <iostream> #include <stdio.h> using namespace std; using namespace cv; int main(int argc, const char** argv) { //create cascade classifier object used face detection cascadeclassifier face_cascade; //use haarcascade_frontalface_alt.xml library face_cascade.load("haarcascade_frontalface_alt.xml"); //setup video capture device , link first capture device videocapture capturedevice; capturedevice.open(0); //setup image files used in capture process mat captureframe; mat grayscaleframe; //create window present results namedwindow("outputcapture", 1); //create loop capture , find faces while (true) { //capture new image frame capturedevice >> captureframe; ...
Read more

The canvas has been tainted by cross-origin data in chrome only -

-
uncaught securityerror: failed execute 'getimagedata' on 'canvasrenderingcontext2d': canvas has been tainted cross-origin data i getting above error in chrome. code(following) works fine in mozilla. have not corss domain issue. why above error in chrome? var wheel_canvas, wheel_context, can_w, can_h, wheel_grd; var large_radius = 160; var small_radius = 120; var wheel_canvas = document.getelementbyid('wheel_canvas'); var wheel_context = wheel_canvas.getcontext('2d'); var can_w = $(wheel_canvas).width(); var can_h = $(wheel_canvas).height(); var centerx = can_w / 2, centery = can_h / 2; var wheel_grd = null; test('#ff0000'); function test(hex) { $('#clrpicker').css({'left': 210, 'top': 210 }); inverted = hexbw(hex); $('#color_val_show').val(hex); current_color_hex_val_selected = hex; $('#color_val_show').css({'color':inverted,'background':hex}); fillgradient...
Read more