D3Js Take Data from an Array Instead of a File

D3js take data from an array instead of a file

The D3 gallery has a lot of good examples, but a lot of the examples loads their data from a tab-separated values file. This is a nice way to separate data and visualization in short examples, but it can be a bit confusing if you are new to D3 and JavaScript, since it requires some basic knowledge about both d3.tsv() and callback functions to understand what is going on.

Disclaimer: The following sections will give some very simplified explanations.

What is d3.tsv() doing?

d3.tsv() is basically responsible for loading the data from data.tsv, parse it into a variable called data, and send this variable to a callback function.

d3.tsv("data.tsv", type, function(error, data) { <- This is the callback function!
// This code is executed when the data.tsv file is loaded.
});

As soon as the data is loaded, it is sent as the data argument to the function. Then the code inside the callback function is executed. Since we don't need the callback function for anything else than as an argument to the d3.tsv() function, we make it directly as an anonymous function, instead of giving it a name like usual.

This kind of use of callback and anonymous functions is very typical of JavaScript, and is well worth reading up on. Understand JavaScript Callback Functions and Use Them and Understanding JavaScript Callbacks should get you started.

Let's see how we can use this information to rewrite our code.

How can I rewrite the example to use data from a local variable?

First we have to make variable containing our data. Let's call it "data", the same as in our callback function, and give it the values from the example.

var data = [
{letter: "A", frequency: .08167},
{letter: "B", frequency: .01492},
{letter: "C", frequency: .02780},
{letter: "D", frequency: .04253},
{letter: "E", frequency: .12702},
{letter: "F", frequency: .02288},
{letter: "G", frequency: .02022},
{letter: "H", frequency: .06094},
{letter: "I", frequency: .06973},
{letter: "J", frequency: .00153},
{letter: "K", frequency: .00747},
{letter: "L", frequency: .04025},
{letter: "M", frequency: .02517},
{letter: "N", frequency: .06749},
{letter: "O", frequency: .07507},
{letter: "P", frequency: .01929},
{letter: "Q", frequency: .00098},
{letter: "R", frequency: .05987},
{letter: "S", frequency: .06333},
{letter: "T", frequency: .09056},
{letter: "U", frequency: .02758},
{letter: "V", frequency: .01037},
{letter: "W", frequency: .02465},
{letter: "X", frequency: .00150},
{letter: "Y", frequency: .01971},
{letter: "Z", frequency: .00074}
];

Put this variable somewhere before the call to d3.tsv, as the code in the callback function is dependent on this variable.

I have choose to represent the data as a list of objects with a letter and frequency property. This is an easy way to do it, since it closely resembles the way d3.tsv() would parse the .tsv file. This means that we don't have to change the code in the callback function, since it already expects a variable with data in this format. You can change this if you like, but remember to change how the callback code uses the "data" variable.

Now we can remove the code related to the d3.tsv call, just leaving the code contained in the callback function. So this code:

d3.tsv("data.tsv", type, function(error, data) {
x.domain(data.map(function(d) { return d.letter; }));
// code omitted.
.on('mouseout', tip.hide)
});

Becomes this code:

x.domain(data.map(function(d) { return d.letter; }));
// code omitted.
.on('mouseout', tip.hide)

Now the example should be working fine. You can play around with this strategy for rewriting other examples in the D3 gallery as well.

Finally, I have included the code for the new index.html file. A working example can be found at JSFiddle.

<!DOCTYPE html>
<meta charset="utf-8">
<style>

body {
font: 10px sans-serif;
}

.axis path,
.axis line {
fill: none;
stroke: #000;
shape-rendering: crispEdges;
}

.bar {
fill: orange;
}

.bar:hover {
fill: orangered ;
}

.x.axis path {
display: none;
}

.d3-tip {
line-height: 1;
font-weight: bold;
padding: 12px;
background: rgba(0, 0, 0, 0.8);
color: #fff;
border-radius: 2px;
}

/* Creates a small triangle extender for the tooltip */
.d3-tip:after {
box-sizing: border-box;
display: inline;
font-size: 10px;
width: 100%;
line-height: 1;
color: rgba(0, 0, 0, 0.8);
content: "\25BC";
position: absolute;
text-align: center;
}

/* Style northward tooltips differently */
.d3-tip.n:after {
margin: -1px 0 0 0;
top: 100%;
left: 0;
}
</style>
<body>
<script src="http://d3js.org/d3.v3.min.js"></script>
<script src="http://labratrevenge.com/d3-tip/javascripts/d3.tip.v0.6.3.js"></script>
<script>

var margin = {top: 40, right: 20, bottom: 30, left: 40},
width = 960 - margin.left - margin.right,
height = 500 - margin.top - margin.bottom;

var formatPercent = d3.format(".0%");

var x = d3.scale.ordinal()
.rangeRoundBands([0, width], .1);

var y = d3.scale.linear()
.range([height, 0]);

var xAxis = d3.svg.axis()
.scale(x)
.orient("bottom");

var yAxis = d3.svg.axis()
.scale(y)
.orient("left")
.tickFormat(formatPercent);

var tip = d3.tip()
.attr('class', 'd3-tip')
.offset([-10, 0])
.html(function(d) {
return "<strong>Frequency:</strong> <span style='color:red'>" + d.frequency + "</span>";
})

var svg = d3.select("body").append("svg")
.attr("width", width + margin.left + margin.right)
.attr("height", height + margin.top + margin.bottom)
.append("g")
.attr("transform", "translate(" + margin.left + "," + margin.top + ")");

svg.call(tip);

// The new data variable.
var data = [
{letter: "A", frequency: .08167},
{letter: "B", frequency: .01492},
{letter: "C", frequency: .02780},
{letter: "D", frequency: .04253},
{letter: "E", frequency: .12702},
{letter: "F", frequency: .02288},
{letter: "G", frequency: .02022},
{letter: "H", frequency: .06094},
{letter: "I", frequency: .06973},
{letter: "J", frequency: .00153},
{letter: "K", frequency: .00747},
{letter: "L", frequency: .04025},
{letter: "M", frequency: .02517},
{letter: "N", frequency: .06749},
{letter: "O", frequency: .07507},
{letter: "P", frequency: .01929},
{letter: "Q", frequency: .00098},
{letter: "R", frequency: .05987},
{letter: "S", frequency: .06333},
{letter: "T", frequency: .09056},
{letter: "U", frequency: .02758},
{letter: "V", frequency: .01037},
{letter: "W", frequency: .02465},
{letter: "X", frequency: .00150},
{letter: "Y", frequency: .01971},
{letter: "Z", frequency: .00074}
];

// The following code was contained in the callback function.
x.domain(data.map(function(d) { return d.letter; }));
y.domain([0, d3.max(data, function(d) { return d.frequency; })]);

svg.append("g")
.attr("class", "x axis")
.attr("transform", "translate(0," + height + ")")
.call(xAxis);

svg.append("g")
.attr("class", "y axis")
.call(yAxis)
.append("text")
.attr("transform", "rotate(-90)")
.attr("y", 6)
.attr("dy", ".71em")
.style("text-anchor", "end")
.text("Frequency");

svg.selectAll(".bar")
.data(data)
.enter().append("rect")
.attr("class", "bar")
.attr("x", function(d) { return x(d.letter); })
.attr("width", x.rangeBand())
.attr("y", function(d) { return y(d.frequency); })
.attr("height", function(d) { return height - y(d.frequency); })
.on('mouseover', tip.show)
.on('mouseout', tip.hide)

function type(d) {
d.frequency = +d.frequency;
return d;
}

</script>

The use of a data array in the d3.js scatter file instead of an external data file

You can use array of hash:

var data = [{"x":"5","y":"90"},{"x":"25","y":"30"},{"x":"45","y":"50"},
{"x":"65","y":"55"},{"x":"85","y":"25"}];

Or write a function to convert your array of arrays to this array of hash.

var rows = new Array(
Array(0,0),
Array(90,90),
Array(59,70),
Array(65,77),
Array(85,66)
);

for (var i = 0; i < rows.length; i++) {
data.push({x: rows[i][0], y: rows[i][1]});
}

// Coerce the strings to numbers.
data.forEach(function(d) {
d.x = +d.x;
d.y = +d.y;
});

Here's the live code:

http://vida.io/documents/THpmgQDARSWPJqGG5

how to access specific data in an array using D3.js v5 for a stacked bar graph tooltip

To get the value from the ith bar, there is actually a second argument, next to d, that you can access in all d3 functions. i denotes the index of that element in the array you passed it.

I changed the mouseover function to just use i here, and it resolved your issue:

const canvas = d3.select('#stacked');

const stackedSvg = canvas.append('svg')
.attr('preserveAspectRatio', 'xMinYMin meet')
.attr('viewBox', '0 0 900 500')
.classed('svg-content', true);

let margin = {top: 40, right: 30, bottom: 20, left: 70},
width = 260 - margin.left - margin.right,
height = 250 - margin.top - margin.bottom;

// append the svg object to the body of the page
let svg = stackedSvg.append('g')
.attr('width', width)
.attr('height', height)
.attr('transform', `translate(${margin.left}, ${margin.top})`);

//number formater
const formater = d3.format(',d');

let myData = [
{
"group": "Field 1",
"0-45": 12345,
"46-65": 91568,
"66-85": 13087,
"85+": 7045
},
{
"group": "Field 2",
"0-45": 62345,
"46-65": 15347,
"66-85": 37688,
"85+": 7007
},
{
"group": "Field 3",
"0-45": 11457,
"46-65": 28456,
"66-85": 124564,
"85+": 7564
},
{
"group": "Field 4",
"0-45": 19234,
"46-65": 26754,
"66-85": 14153,
"85+": 7453
}
]

//tooltip
let tooltip = d3.select('body').append('div')
.attr('class', 'tooltip')
.style('opacity', 0);

// Parse the Data
Promise.resolve(myData)
.then(data => {
// console.log(data);

//select the size bands
let keys = d3.keys(data[0]).slice(1);
// console.log(keys);

// List of groups in JSON. value of the first column called group
let groups = d3.map(data, function(d){return(d.group)}).keys();
// console.log(groups);

// X axis
let x = d3.scaleBand()
.domain(groups)
.range([0, width])
.padding([0.2])
svg.append('g')
.attr('transform', 'translate(0,' + height + ')')
.call(d3.axisBottom(x)
.tickSizeOuter(0));

svg.append('text')
.attr('class', 'xLabel')
.attr('text-anchor', 'end')
.attr('x', width / 2 + 20)
.attr('y', height + 30)
.text('Size Band')
.style('font-size', 10);

// Y axis
let y = d3.scaleLinear()
.domain([0, d3.max(data, d => d3.sum(keys, k => +d[k]))])
.range([ height, 0 ]);

svg.append('g')
.call(d3.axisLeft(y)
.tickSizeOuter(0)
.tickSizeInner(- width)
.tickPadding(5))
.selectAll(".tick")
.style("stroke-dasharray", ("1, 1")) //dash line across graph.
.each(function (d, i) {
if ( d == 0 ) {
this.remove();
}
});

svg.append('text')
.attr('class', 'yLabel')
.attr('transform', 'rotate(-90)')
.attr('y', - 40)
.attr('x', - height / 2 + 20)
.attr('text-anchor', 'end')
.text('Units')
.style('font-size', 10);

// color
let color = d3.scaleOrdinal()
.domain(keys)
.range(['brown', 'steelblue', 'olivedrab', 'darkorange']);

//stack the data --> stack per subgroup
let stackedData = d3.stack()
.keys(keys)
(data);
console.log(stackedData)

let totalColVal = d3.nest()
.key(function(d){return (d.group)})
.rollup(function(totals){
return d3.sum(totals, function(d) {return d3.sum(d3.values(d))})
}).entries(data)

console.log(totalColVal)

//tooltip
let mouseover = function(d, i) {
let subgroupName = d3.select(this.parentNode).datum().key;
// console.log(subgroupName)
let subgroupValue = d.data[subgroupName];
// console.log(subgroupValue)

tooltip.html('<b>Field:</b> <span style="color:yellow">' + d.data.group + '</span>' + '<br>\
' + '<b>Count:</b> <span style="color:yellow">' + formater(subgroupValue) + '</span>' + '<br>\
' + ' <b>Size Band:</b> <span style="color:yellow">' + subgroupName + '</span>' + '<br>\
' + '<b>Field Total:</b> <span style="color:yellow">' + totalColVal[i].value + '</span>')
.style("opacity", 1)
};
let mousemove = function(d) {
tooltip.style('left', (d3.event.pageX - 70) + 'px')
.style('top', (d3.event.pageY - 85) + 'px')
};
let mouseleave = function(d) {
tooltip.style("opacity", 0)
};

// Show the bars
svg.append('g')
.selectAll('g')
// Enter in the stack data = loop key per key = group per group
.data(stackedData)
.enter().append('g')
.attr('fill', function(d) { return color(d.key) })
.selectAll('rect')

// enter a second time = loop subgroup per subgroup to add all rectangles
.data(function(d) { return d; })
.enter().append('rect')
.on('mouseover', mouseover)
.on('mousemove', mousemove)
.on('mouseleave', mouseleave)
.transition()
.attr('y', d => y(d.height))
.delay(function(d, i) {
return i * 100
})
.ease(d3.easeLinear)
.attr('x', function(d) { return x(d.data.group); })
.attr('y', function(d) { return y(d[1]); })
.attr('height', function(d) { return y(d[0]) - y(d[1]); })
.attr('width',x.bandwidth())
.style('stroke', 'black')
.style('opacity', 0.9);

});
body {
font-family: halyard-display, sans-serif;
/* background-color: black; */
}

div.tooltip {
position: absolute;
text-align: left;
width: fit-content;
height: fit-content;
padding: 5px 5px;
font-size: 16px;
background:rgb(24, 23, 23);
color: white;
border-radius: 5px;
pointer-events: none;
}

.yLabel {
fill: #DEDC00;
font-style: italic;
font-weight: 600;
}

.xLabel {
fill: #DEDC00;
font-style: italic;
font-weight: 600;
}

g .tick text {
font-size: 8px;
color: grey;
}

g .tick text:hover {
font-size: 12px;
}

g .tick {
color: #DEDC00;
}
<script src="https://d3js.org/d3.v5.js"></script>
<div id="stacked"></div>

d3 importing csv file to array

In d3 v5 the API for fetching data has changed quite a bit, which became necessary as the underlying workings have switched from using XMLHttpRequest to the Fetch API. In prior versions of D3 up to v4 your code would have behaved as you expected printing the single resulting array. The new API for d3.csv(), however, look like this:

# d3.csv(input[, init][, row]) <>

Further down the docs provide an explanation for your observation:

If only one of init and row is specified, it is interpreted as the row conversion function if it is a function, and otherwise an init object.

In your code the second argument to d3.csv() is a function and is, thus, interpreted as the row conversion function. Because the row conversion function is executed for every single row in your input file you see each object printed individually instead of the whole array at once.

Since d3.csv() returns a Promise the correct usage would be like this:

d3.csv("data.csv")
.then(function(data) {
console.log(data);
});

Here data refers to the entire array of objects.

How do I read a CSV file into an array using d3js v5

When you run your code you should see that you are indeed loading the data. When I use your example data in a dsv/csv file and run your function I get the following logged to console:

Sample Image

So it appears as both your load of the dsv file is successful, and your row function is successful, based on the console log in the then method.

Let's take a closer look at the .then method, this runs after the file has loaded and the row function been applied - or, once the promise has been fulfilled. So the key part here is the then method doesn't execute the provided function until after the promise is fulfilled. While we are waiting for the the promise to be fulfilled, code continues to be executed, in your case the return statement executes - before the data is fully loaded and therefore before the then method executes. This means you aren't returning any data, and this is your problem.

The simplest and probably most common d3 patterns to fetch the data with your row function is to either place code that requires the loaded data in the .then method's fulfillment function:

function getCSVdata( ) {    d3.dsv(";","https://raw.githubusercontent.com/Andrew-Reid/SO/master/dsv.csv", function(d) {        return {            State : d.Country,            freq: {                 low  : d.Car,                med  : d.Van,                high : d.Truck            }        };    }).then(function(data) {      console.log("file has " + data.length + " rows");      logger(data);    }); }
getCSVdata();
function logger(data) { console.log(data);}
.as-console-wrapper { max-height: 100% !important; top: 0; }
<script src="https://d3js.org/d3.v5.min.js"></script>

how to get data with tsv or csv to array in d3.js from a txt file?

SOLVED

TSV doesn't just take tab seperated elements a throw them into an array. You the format to look like this:

Group1 Group2 Group3
data1 data1 data1
data2 data2 data2
data3 data3 data3

Then you can print out and verify that the data is loaded:

data.forEach(function(d) { 
console.log(d);
});

When printing out the entire data D3 will create an objects that each have a Group bound with all of it's data. Here's the output of my example:

Object {Group1: "data1", Group2: "data1", Group3: "data1"}
Object {Group1: "data2", Group2: "data2", Group3: "data2"}
Object {Group1: "data3", Group2: "data3", Group3: "data3"}

Those the objects are equivalent to data[0], data[1] and data[2], which leaves group1, group2 and group3.

d3.js parse JSON-like data from file

As @Shashank points out your JSON is simply invalid. This has nothing to do with d3 but rather JavaScript and acceptable JSON formats. So, how can we fix it? You could do it in your d3/javascript itself instead of pre-processing the files.

// how many lines of JSON make an object?
var chunks = 5,
data = [];
//download file as TEXT
d3.text("data.txt", function(error, text) {
if (error) throw error;
// split on new line
var txt = text.split("\n");
// iterate it
for (var i = 0; i < txt.length; i += chunks) {
var block = "";
// grab blocks of 5 lines
for (var j = 0; j < chunks; j++ ){
block += txt[i+j];
}
// parse as JSON and push into data ARRAY
data.push(JSON.parse(block));
}
console.log(data);
});

Here it is running on a plunker.

Make a line chart in D3.js loading data from an array

You have another date format. Try this:

var parseDate = d3.time.format("%d-%m-%Y").parse;

And use d.date instead of d.time in your x.domain:

x.domain(d3.extent(data, function(d) { return d.time; }));

Demo: https://embed.plnkr.co/SGRyjWxjReXq8ENLd4LI/



Related Topics



Leave a reply



Submit